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THE QUESTIONNAIRE 





Have you dispatched a carefully prepared questionnaire, ask- 
ing for significant data? 


Has this questionnaire gone to a group of persons who ordi- 
narily treat you with courtesy and respect? 


My) 
15 
& 
i) 
0) 
is) 
Ny) 
(5) 
5 
(5) 
i) 
C5 
Wy 
as 
9] 
ry 
o 
5 
wy 
is 
6 


Has your questionnaire been thrown into the waste basket 
with the others—good, bad, and indifferent—which roam the coun- 
try unrestrained? 
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Have you neglected pressing duties, and burned midnight oil 
while you answered questionnaires? 


Have you anathematized questionnaire-makers and all their 
works? 
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Have you purchased a large waste basket and resolved to an- 
swer no more questionnaires, question-blanks, check lists, in- 
quiries, or data sheets? 
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If you have done any or all of these things—this Bulletin is 
for you. 
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FOREWORD 


HREE characteristics distinguish fruitful educational investigation. First, the 
‘Tae of inquiry is timely. Second, discrimination is exercised as to how it is 
investigated. ‘Third, results are presented in a form likely to affect practice con 
structively. This issue of the Bulletin abundantly fulfills each of these criteria. 


That the questionnaire is widely used in educational investigation is obvious. 
That its use has troubled many educators both in schools and colleges is equally clea: 
That more heat than light has been generated by some discussions of the “questionnaire 
problem” is testified by references cited in the bibliography. An investigation of the 
problem is therefore timely. 


Discrimination has been used in selecting proper methods of studying this difficult 
problem. Enough data have been presented, but not too much. Both the sins as wel! 
as the virtues of this “wayward, but indispensable” tool of science are laid bare. 


Finally, a constructive program is proposed. Those who would “do something’ 
about questionnaires need not hesitate as to what to do if they will carefully stud 
the material made available herein. It offers a plan which is both sound and workable. 


One warning is desirable. The entire Bulletin should be read before final con- 
clusions are drawn. Material presented in certain sections reveals the questionnaire 
at its worst. Facts in other sections bring out the contribution it has made to educa- 
tion. The significance of certain paragraphs will be changed if they are taken from 
their context. If the central finding were to be stated, it would take some such word- 
ing as this: 


The abuses which have grown out of the circulation of questionnaires can be corrected by 
voluntary, cooperative regulation. - For those who receive questionnaires this involves the 
inauguration of plans whereby worthy inquiries may be selected from the rest and answered 
with the least expenditure of energy. For those who send questionnaires, it calls for planning 
to the ends that no unnecessary nor improperly prepared inquiries may be circulated. It is 
important that these means of correction be taken. Otherwise one of the methods of investiga- 
tion which has contributed substantially to educational advance will be cut off at the source. 


This Bulletin is recommended to those who would exert a constructive influence 
in the regulation of the questionnaire problem. It will be of interest to superintend- 
ents of schools and their staffs, college specialists, graduate students of education, and 
others who directly or indirectly come in contact with questionnaire investigation. 


J. W. Crasrree, Secretary, 


National Education Association. 
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Introduction 


ORACE MANN in 1847 sent a ten 
page “Circular” asking certain teachers 
in Massachusetts a series of questions. 

It began as follows: 


I desire to obtain the opinion of teachers who are 
both scientific and practical on a subject of great 
importance to the cause of popular education." 


He little realized that in sending this and 
subsequent Circulars to 
vated character, and whose extended personal 
acquaintance with the subject matter on which 
they testify, place them above denial, cavil, or 
suspicion” he was creating a precedent later to 
be followed by so many as to reach the propor- 
tion of a “problem.” 

The inception of this Bulletin—Recent years 
have witnessed a rapid growth in the use of the 
questionnaire as a means of educational investi- 
gation. Figures presented in a later section in- 
dicate that several hundred different question- 
naires are annually circulated among the of- 
fices of superintendents of schools in this coun- 
try, and that some individual school systems re- 
ceive more than a hundred each year. 

The work of answering questionnaires has 
come to be regarded as an unjustifiable burden 
by some. Magazine articles anathematizing the 
questionnaire-makers and all their works have 
appeared. Some school systems have taken 
blanket action against answering all such in- 
quiries. 

The members of the Department of Super- 
intendence of the National Education Asso- 
ciation have repeatedly, as individuals and as 
an organization, taken notice of the “question- 
naire problem.”’ When the original proposal to 
establish the Educational Research Service was 
made seven years ago by R. G. Jones,? a prin- 
cipal motive was to reduce the number of ques- 
tionnaires by establishing a “clearing house”’ 
which would make many of them unnecessary. 
Official notice was again taken of this problem 
in the resolutions of the 1929 meeting of the 
Department of Superintendence. 

Questionnaires have become so unpopular 
that many who send them avoid the word. In- 


oe 
persons whose ele- 


! Life and Works of Horace Mann. Lothrop, Lee and She ms “rd Co., 


? Jones, Robinson G. ‘‘A Program for the Coming Year. 
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stead they issue “inquiry forms,’’ “check-lists,” 


and “data sheets’ accompanied by apologetic 
notes. : 
These conditions stimulated the Research 


Division to study the whole problem. The col- 
lection of material bearing upon it began sev- 
eral years ago and an intensive study of some 
phases of the question has been under way for 
the last two years. 

The purpose of this Bulletin—This issue of 
the Research Bulletin presents material perti- 
nent to the proper solution of the problem. The 
aim is to answer these questions: 

1. Do questionnaires serve useful purposes, and 

if so, what are they. (See pages 11-3.) 

- How may questionnaires be better prepared? 
(See pages 14-24.) 

3. How may the number of questionnaires be re- 
duced? (See pages 14-7, and 38-43.) 

4. How may the results of questionnaires be made 
more useful? (See pages 25-7.) 

5. What is the size and character of the problem 
of dealing with questionnaires as it is en- 
countered by superintendents of schools? (See 
pages 28-37.) 

6. What should be the attitude and practice of 
superintendents of dealing with 

questionnaires? (See pages 13, 38-43.) 

- How may the various interested agencies co- 
operate in an intelligent regulation of the ques- 
tionnaire problem? (See pages 38-43.) 


nN 


schools in 


“I 


General conclusions—The outcome of in- 
vestigations of the questionnaire problem which 
this Bulletin presents may be epitomized in 
these words: 

The practice of circulating questionnaires in 
the field of education has undoubtedly been 
abused. The time of busy people is sometimes 
wasted by time-consuming questionnaires deal- 
ing with inconsequential topics, worded so as 
to lead to worthless replies, and circulated by 
untrained and inexperienced individuals, lack- 
ing in facilities for summarizing and dissemi- 
nating any worthwhile information which they 
may obtain, 

From these facts, however, one should not 
draw the conclusion that the questionnaire 
should be abolished. There are competent in- 


Boston, Mass., Vol. IV, page 151. 


Addresses and Proceedings of the National Education Association, 
1923, pages 907 to 909. For information concerning the value and cost of the Educational Research Service 
ment of Superintendence, National Education Association, 1201 16th St. N. W. 


address the Depart- 
, Washington, D. C. 





Figure I 


QUESTIONNAIRES «xd POOR ADMINISTRATION 


Is this the way you handle questionnaires ? 











Superintendent. of Schools: These 
questionnaires are & nuisance. 


I have no time to answer y= 
them . grr 
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Superintendent of Schools: How can 

| advise my board when there are no facts ? 
Director of Research . Sorry 

sir, but you know we never an- Zp 

swer questionnaires and in- 

vestigators have no other way 

to get such information x 




















Figure II 





- QUESTIONNAIRES «4 GOOD ADMINISTRATION 








Do you handle questionnaires this way ? 





Superintendent of Schools : Have this Y \ 
questionnaire evaluated and if* it eo 


deserves attention see that the inform od 


ation requested is furnished. oN 


N V 








Is this what happens when you want school facts ? 
Superintendent of Schools: Excellent. ! 


This is just the information I need in A 
advising my board. CAB ,.~ _ 
Director of Research: The factscame \ * 4) .t 





from €@ questionnaire. \ 
study in which we 
recently cooperated. 
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vestigators who issue carefully prepared ques- 
tionnaires on vital educational questions and 
make the results generally available in usable 
form, 

A broad and unbiased study of the question- 
naire movement as a whole leads to the con- 
clusion that this method of investigation, in 
spite of its abuse, has yielded far more on the 
credit than on the debit side of educational ad- 
vance. Much of the significant knowledge we 
now have in the field of education has come 
from the questionnaire in the hands of qualified 
investigators, and for some time to come it will 
undoubtedly constitute a legitimate means of 
educational investigation and research. Super- 
intendents of schools and their staffs, professors 
of education, and graduate students of educa- 
tion, to name those who most often originate 
questionnaires, find them indispensable, and 
they will continue to circulate them. This be- 
ing the case, what should be the attitude of one 
who wishes to exercise an enlightened influence 
on this problem? 

If he is one who circulates questionnaires he 
will examine material presented in, or cited by, 





1 When references are not given in complete form in the footnotes, they will be found fully cited in the bibliography on pages 


47-9, 
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this Bulletin * which will help him to prep.re 
questionnaires well, which will guard him fro.) 
sending unnecessary questionnaires, and w!| 
will aid him in making the replies to questi: 
naires most useful to himself and others. 

If he is one who receives questionnaires 
will examine the material presented in, or ci: 
by this Bulletin which will aid him in 
tinguishing between worthy and unwort! 
questionnaires, and to set up a simple, continy 
ing provision for economically answering those 
which are worthy. 

Finally, whoever he is, school administrat 
research worker, or teacher, he will recognize 
that it is the duty of every one to do what hie 
can to advance educational knowledge, whethe: 
his contribution be large or small, direct or in 
direct. He will strive to make the energy ex 
pended in meeting this responsibility count for 
most. He will not waste the time of others }) 
circulating poorly prepared, unnecessary in 
quiries, on inconsequential topics, nor wil! he 
allow his own time to be wasted by answering 
such inquiries. 
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Questionnaires—To Be or Not To Be? 


There is ample evidence that the question- 
naire has fallen into disfavor with many. Some 
would even go so far as to issue a blanket in- 
terdict against this method of investigation. 
What justification is there for such an atti- 
tude? Let us examine the evidence both against 
and in favor of this method of inquiry. 


The Case Against The Questionnaire 

These are the objections frequently raised 
against this method of assembling data: 

1. The results of most questionnaires make 
little if any contribution to educational ad- 
vance. First-rate educational investigations do 
not employ this method. Conclusions based on 
questionnaire blanks are seldom final. 

2. The respondents to a questionnaire may 
not be a random sampling of those addressed. 
Selection may invalidate the representative- 
ness of data, particularly if the percent of reply 
is small. 

3. Respondents to a questionnaire may con- 
sciously or unconsciously modify their replies 
to give the investigator what he wants. 

4. Carelessness in reporting, difficulty of 
securing information requested, and other fac- 
tors favor inaccuracy in replies to question- 
naires. The degree of this inaccuracy cannot be 
estimated. 

5. The questionnaire method is indefensibly 
*time-consuming particularly for the respondent. 

6. Questionnaires are frequently poorly pre- 
pared, ask for information already available, 
and the results are often not properly summar- 
ized and distributed. 

These objections deserve serious considera- 
tion. Are they valid to the degree which jus- 
tifies the abolition of the questionnaire as a 
tool of the serious educational investigator? 
Each will be considered in turn. 

Much effort for inconclusive results—There 
is evidence that some questionnaires contribute 
little if anything to educational advance. The 
Research Division made an intensive follow- 
up of 267 questionnaires on a variety of topics 
received during 1927-1928 by the offices of 
certain superintendents of schools. No report 
of any kind could be obtained as to what hap- 
pened to 64 of the 267. A considerable per- 





1 See page 36-7 of this Bulletin. 
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centage of the questionnaires for which reports 
could be obtained were not tabulated or used 
in any way. Less than half of them eventuated 
in either a mimeographed or printed report 
which could be circulated among respondents. 
It appears that many questionnaires make little 
contribution, even to the one circulating them. 
These facts, however, do not justify the con- 
clusion that no first-rate contributions result 
from questionnaires. Conclusive evidence to the 
contrary is presented later. 

The objection that questionnaire investiga- 
tion seldom conclusively answers the question 
at issue is not a serious one. In modern science 
no answer to any question is final. Many sup- 
posedly final answers to problems in the field 
of physics, 
revised in the last decade, and the final answers 
are still being sought. 

Selection of replies—The danger that the re- 
plies to questionnaires may represent a selected 


a most exact science, have been 


group is a real one. ‘There are undoubtedly 
situations where this factor invalidates conclu- 
sions. This danger is not an insuperable one, 
however, for these reasons: 


a. A well prepared and properly motivated 
questionnaire, with careful follow-up, may 
secure such a high percent of reply that the ad- 
dition of the missing replies could not signifi- 
cantly modify data actually received. For ex- 
ample, 67 of 68 cities over 100,000 in popula- 
tion replied to the extensive salary data blank 
sent out by the Research Division in 1928- 
1929. A reply from the missing city could have 
only slightly affected the general results. 

b. Objective appraisals of this factor such 
as that of Toops, cited in the bibliography on 
page 49, suggest that the danger of a selected 
response has been overestimated. Until evi- 
dence to the contrary is forthcoming, it seems 
reasonable to conclude that this factor is prob- 
ably not decisive in affecting the results of the 
typical questionnaire, particularly if a reply of 
more than 50 percent is received. One can never 
be sure on this point, however, and unless a 
reply of practically 100 percent is obtained it 
is well to heed Kelley’s words of caution: 


. . . lack of demonstrable fairness in the sample 
is probably the greatest shortcoming of the ques- 





tionnaire. The user of the questionnaire should 
never lose sight of it.’ 

Desired answers given—Concerning the ob- 
jection that respondents tend to give answers 
desired by the interrogator, there are doubtless 
situations in which this is a factor of signifi- 
cance, particularly when subjective opinions or 
attitudes are called for and leading questions 
are used. It is unlikely that this objection ap- 
plies to most well-prepared questionnaires for 
these reasons: 

a. The investigator often does not, and 
properly should not desire responses, other than 
those which present the truth, and even if he 
does have his private opinions, they are not 
known to the respondents. 

b. Most questionnaires are largely made up 
of questions of fact, answerable with yes, or 
no, or other objective reply, and reduce the 
number of questions calling for a subjective 
reply to a minimum. Hence, the probability 
that answers are modified so as to suit the in- 
vestigator is remote. 

Data fundamentally inaccurate—The ob- 
jection that the questionnaire inherently en- 
courages careless reporting, and therefore se- 
cures inaccurate data, is a serious one. It has 
also been pointed out that respondents are 
either unable or unwilling to give accurate an- 
swers to certain types of questions calling for 
personal or confidential data. 

These are unanswerable indictments if small 
differences are significant and extreme nicety in 
original data is essential. There are many edu- 
cational questions, however, in which an ap- 
proximately correct measure of central tend- 
ency, or a general indication of trends, is suffi- 
cient for the purpose. The sometimes over- 
worked, but none the less valid statistical prin- 
ciple that chance errors tend to neutralize each 
other, particularly in influencing general aver- 
ages, also applies here. 

An alert investigator will discover means of 
checking at least portions of his data. For ex- 
ample, rechecks of data secured by the salary 
inquiry of the National Education Association, 
made in connection with intensive local salary 
investigations, indicate these data to be sufh- 
ciently exact so that not only the ranks, but the 
absolute position of individual cities may be 
designated with reasonable confidence. 





1 Kelley, Truman L. Scientific Method, page 81. 
* See page 33 of this Bulletin. 


Education is still far enough from being 
exact science so that data assembled by 
method inherently involves a degree of e: 
often unmeasured, if recognized at all. I{ 
questionnaire is ruled out on this basis p: 
cally all other methods of educational iny. 
gation deserve a similar fate. 

Time-consuming—The objection that que: 
tionnaires are indefensibly time-consumin 
particularly of the respondent’s time, is va 
in too large a percent of cases. This, howeve: 
is not an inherent objection. An intelligent 
lection of only worthy questionnaires for atten 
tion, and proper organization of such time an 
facilities as one has to give to them, wil! dis 
courage the circulation of inquiries which ar 
unnecessarily time-consuming. 

Poorly prepared and handled—That unr 
essary, poorly prepared questionnaires are ci: 
culated by people who lack facilities for sum 
marizing and disseminating results may be con 
cluded from facts presented elsewhere. The in- 
vestigation upon which this Bulletin is based 
however, did not reveal a serious amount of 
overlapping among questionnaires.” It is tru: 
that a considerable number on a common sub 
ject may be issued each year. An analysis of 
these inquiries, however, reveals that they fre 
quently deal with different phases of the same 
general topic, and the amount of overlapping 
of specific questions is small. It appears that the 
amount of duplication represented by ques 
tionnaire investigations has been over-emphia 
sized. 

An appraisal of the objections—A review ot 
the foregoing objections to questionnaires re- 
veals none that justify blanket condemnation 
of this method of assembling data. Most of 
them consist of criticisms of the abuse of, 
rather than the right use of, the questionnaire. 

The admitted fact that many questionnaires 
contribute little or nothing, loses much of its 
significance when weighed with equally in- 
dubitable evidence that some questionnaires 
contribute a great deal. The situation is not 
unique. Any form of educational investigation 
works much ore for a little gold. The lessons 
from this fact, as it affects questionnaires, 
should be—start with better ore, and refine it 
by better methods. Practical means for doing 
this are suggested subsequently. 


[10] 





Other important objections to the question- 
naire method—that a selected group of re- 
spondents may reply, and that it inherently en- 
courages inaccurate reporting—lose their po- 
tency except when applied to poor execution 
‘qn using this method of investigation. If it ap- 
ears, because of the character of the informa- 
sion needed and the composition of the groups 
which can supply it, that a selected reply would 
be likely to be received, then the questionnaire 
probably should not be issued. If, however, a 
questionnaire is issued under such circum- 
stances, follow-ups should be sent out until a 
suficient percent of reply is received so that 
the factor of selection is minimized. Realizing 
that a degree of inaccuracy may characterize 
questionnaire replies, the competent investi- 
vator will either not use this method when 
exact data are requisite, or will provide a 
method of validating results, as Koos? sug- 
gests. 

The difficulties which grow out of the more 
obvious and flagrant abuses of the question- 
naire by untrained and inexperienced people 
are not insuperable. Inexperienced, even vicious 
people, sometimes misuse the automobile, but 
no general outcry is raised against this method 
of transportation. In this, as in the question- 
naire problem, cooperation in regulation is the 
solution. A careful search of available liter- 
ature criticizing the use of questionnaires re- 
veals nothing, either in the form of theoretical 
argument or of objective evidence, which justi- 
fies a sweeping condemnation and complete 
abolition of the questionnaire as a tool of edu- 
cational investigation. 

Let us next consider the case for the ques- 
tionnaire. The case against this method of in- 
quiry, particularly when confined to its use by 
competent investigators is not a serious one. 
But what can be said in favor of the question- 
naire? 


The Case For The Questionnaire 

“By their fruits ye shall know them” has 
many applications. Has the questionnaire 
vielded significant fruits in the form of con- 
tributions to educational knowledge and prac- 


tice ? 


* Ibid. page 57-68. 

* Kelley, Truman L. Scientific Method, page 43. 
‘ Ibid. page 50. 

‘Ibid. page 83. 


Koo’s recent book * presents important evi- 
dence on this question. His study analyses 
581 representative, published 
the field of education. Of these 581 studies, 
143 or nearly one-fourth, depend wholly or 
principally on questionnaire data. Only two of 
the 143 were found to have “little or no value.” 
Koos concludes that the question blank has 
proved to be a source of significant educa- 
tional data, and that many valuable studies in- 
cluding some of exceptional merit have made 
use of this method. 

Kelley * recently approached the 
from a different angle. He asked a small group 
of college scientists in a variety of fields the 


researches in 


matter 


following question: 


In your opinion is a questionnaire ever an appro- 
priate instrument for use in the conduct of serious 
research in any scientific field? 

From seventy-five to a hundred percent of 
each of eight scientific groups catechized gave 
at least a qualified yes to the foregoing ques- 
tion.* Kelley closes thus: 

Let me conclude this investigation of the value of 
the questionnaire by saying that unless and until 
experimental science relieves us of the need of 
human judgments, or removes from our minds an 
interest in unique events, this wayward child of 
science, feeble as it is, will remain an indispen- 
sable helper. It will thus be always needed, and we 
can but hope that it will curb its intrusive dis- 
position and mend its unseemly ways.* 

Illustrative questionnaire investigations. A 
moment’s thought will convince those in touch 
with educational investigation, that many sig- 
nificant studies lean heavily upon data obtained 
by questionnaire. 

Counts has published several investigations 
of recognized merit largely based on question- 
naire results. His study of boards of educa- 
tion ° is a recent example. After making gen- 
erous allowance for a certain degree of ad- 
mitted inaccuracy of the data, one accepts this 
investigation as reliable. It presents the most 
complete analysis of the make-up of school 
boards made to date and raises a series of fun- 
damental questions concerning their composi- 
tion. It would have been impractical to have 
undertaken this study except by questionnaires. 


' Koos, Leonard V. The Questionnaire in Education. Macmillan Co., 1928, p. 138. 


. 


* Counts, George S. The Social Composition of Boards of Education. Supplementary Education Monographs, No. 33, Univ. of 


Chicago, Chicago, IIl., July, 1927. 100 pages. 
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Another study based largely on question- 
naire data is that by Buckingham concerning 
teacher supply and demand.' In spite of short- 
comings which the author points out in this 
study, it deserves to rank as a major contribu- 
tion in an important field, and one about which 
the teaching profession will hear much during 
the next decade. 

‘Terman’s studies of genius” represent an 
entirely different field in which considerable 
use was made of questionnaires. Whatever in- 
terpretation one may make of the facts pre- 
sented in this study, and upon whatever side 
one may array himself in the merry battle of 
nature versus nurture, he can hardly fail to 
place this among the more important studies of 
recent years—modifying or over-turning as it 
does many popular fallacies concerning that im- 
portant element of our human resources, the 
gifted child. 

Koos is another well-known investigator who 
has used the questionnaire extensively. A series 
of question blanks constituted an important 
means of assembling the data upon which he 
based his authoritative study of the junior col- 
lege.* Few would question the great value of 
Koos’ investigation of this important move- 
ment. Surveys such as this would either be pro- 
hibitively expensive, or much less comprehen- 
sive, but for the questionnaire. 

Briggs’ early and widely influential book on 
the junior high school * used questionnaire re- 
turns from many schools as one of several 
methods of securing data. 

Charters and Waples used the questionnaire 
as one of several methods of securing data in 
their teacher training study.® This fundamental 
investigation presents a picture of the eval- 
uated activities and traits of certain types of 
teachers and should be of large value in the 
construction of professional courses for teach- 
ers. 

Questionnaires were one of the principal 
tools of inquiry in the recent modern foreign 


1 Buckingham, B. y 
Ohio State ee tlm on Ohio, = 18 
* Terma s M. et al. Genetic Stud: 
Stanford Universit ty. Press, > F.-Y Calif, 1926. 
§See: Leonard V. The Junior Colle, 
ey 5. Minneapolis, Minnesota, 1924. 682 p. 








language study. Volume four ® of this in 
gation, dealing with enrollment in foreign |). 
guages, is largely based on questionnaire re. 
plies. Few will doubt that this inquiry i: oj 
major significance to secondary schoo! an 
lege development in the United States. 
Judd’s study of the length of elemen 
education’ based partly on data collected 
through question blanks arrives at this im, 
tant conclusion: 


Ample evidence is present in the experienc 
the United States and Canada that elementary edy 
cation of satisfactory degree of richness of conten 
can be provided in seven grades. 


The Land Grant College survey now being 
conducted by Arthur J. Klein of the United 
States Office of Education secured much ot its 
data from a series of carefully prepared, but 
none the less long and detailed questionnaires 
When this study becomes available shortly, jr 
will doubtless constitute the most fundamental 
survey of an important section of higher edu- 
cation which has yet been made. 

The foregoing illustrations which might be 
duplicated many times * are sufficient to prove 
that questionnaire investigation does result in 
contributions of major significance. 

The questionnaire as a trail-blazer— \ be 
foregoing paragraphs present evidence as to 
the value of the questionnaire as employed in 
fundamental educational investigation usually 
by the college specialist. It is also used in an- 
other form of investigation which, though not 
as fundamental as careful research, is never- 
theless important. 

The question blank is frequently of par- 
ticular value as a trail-blazer to educational 
investigation. Suppose one is contemplating a 
study, let us say, of the progress which has 
been made in developing differentiated curric- 
ulums to meet the varying aptitudes of second- 
ary school attendants, and his method is to be 
an intensive case study of the content and 


and Demand in Teacher Training.” Bureau of Educational Research Monograph, No. 4. Th 
_ Vol. Le and Physical Traits of a Thousand Gifted Children 
¢. Vols. I and II, Rensasch Publications of the University of Minnesota, Education 


Also see: Koos, Leonard V. The Junior ‘College Movement. Ginn and Company, Boston, 1925. 436 p. 
Thom as H. The Junior High School. ee = 4m Mifflin Co. Boston, 1920. 350 p 
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affect of differentiated curriculums in ten high 
schools. It would be sheer folly to select ten 
high schools at random for this purpose. 
Rather, a brief exploratory questionnaire should 
vo out to a large number of high schools in 
ondes to discover those which have done most 
to develop differentiated curriculums. Then 
the study should go forward in the ten schools 
where conditions promise most for the investi- 
gation. The questionnaire is frequently an ef- 
fective method of locating the veins of ore 
which may be worked with most profit. 

The school superintendent and question- 
naires—Another part of this bulletin considers 
the problem which superintendents of schools 
encounter in dealing with questionnaires. Per- 
tinent here, however, is the fact that superin- 
tendents and their staffs are among the most 
prolific distributors of questionnaires. It is 
likely that they will continue to use them. The 
questionnaire is frequently the only means of 
obtaining certain types of information which 
superintendents of schools and members of 
their staff must have. 

The school executive is the spearhead of 
educational progress so far as practical appli- 
cation is concerned. New issues are constantly 
arising, and he must frequently advise his 
board on short notice concerning a policy about 
which little or nothing is in printed form. 
What is he to do? It is stupid to say that he 
should base his advice solely on his own opin- 
ion, when he may circulate a brief question- 
naire among a few fellow superintendents and 
secure other judgments on the policy at issue. 

It is unlikely that scientific research, or 
logical reasoning—either or both—will ever 
anticipate all of the problems which the prac- 
tical school executive will encounter. Common 
practice and experience will doubtless always 
be one of the guiding forces in educational 
development. The questionnaire is one means 
of tapping this experience. It will, and should 
continue to be employed for this purpose. 

Questionnaires often the only way—Finally, 
there remains the very important consideration 
that there is frequently no practical substitute 
for the questionnaire. Koos concluded that only 
a negligible proportion of the 143 fundamental 
questionnaire researches which he analysed 
could have used any other method of investi- 
gation.’ The same may be said concerning most 
~~ 1 Koos, Leonard V. The Questionnaire in Education. page 64. 


of the carefully prepared questionnaires of the 
trail-blazing type—those which nominate de- 


sirable subjects for more careful investigation. 


Likewise, questionnaires circulated by school 
executives, those which collect information as 
to current practice and secure judgments on 
educational innovations, frequently seek un- 
available information which it is not practical 
to secure otherwise. 

Summary and conclusion—The first part of 
this chapter views the objections raised against 
the use of questionnaires in educational investi- 
gations. It has been stated that this method of 
inquiry requires much effort by the investigator 
for inconclusive results, assembles data that 
are unreliable and inaccurate, places too heavy 
a burden on respondents, and is generally poorly 
applied. Analysis of the foregoing objections 
indicates that they consist mainly of criticisms 
of the abuse rather than the right use of the 
questionnaire, and that there is insufficient basis 
either in the form of theoretical criticisms or 
objective evidence to justify a sweeping con- 
demnation of this instrument of educational 
investigation. 

The case in favor of the questionnaire is 
also examined. Evidence presented indicates 
that questionnaire studies have contributed sub- 
stantially to the store of educational know]l- 
edge. The questionnaire serves a useful pur- 
pose in discovering veins of ore which deserve 
intensive working by research specialists. It 
is also considered an essential tool by many 
school executives in securing information con- 
cerning school practices and procedures. It 
would not be practical to obtain much of the 
information assembled by questionnaire which 
contributes to the foregoing significant out- 
comes through any other means. 

The abolition of this method of inquiry 
would work an unnecessary hardship on the 
scientific investigator in education as well as 
upon the practical school executive. No gen- 
eral abolition is necessary. The abuses which 
have risen out of the circulation of question- 
naires can be corrected by voluntary, coopera- 
tive regulation along lines suggested in a sub- 
sequent chapter. It is important that this 
method of correction be taken. Otherwise, one 
of the methods of inquiry which has substan- 
tially contributed to educational advance will 
be cut off at the source. 
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Better Preparation of Questionnaires 


Elsewhere in this Bulletin it is suggested 
that a principal cause of the disfavor in which 
many hold the questionnaire method of in- 
vestigation is the lack of skill with which it is 
used. This section will suggest certain guides 
for the preparation of questionnaires. These 
guides are based partly upon the experience of 
the Research Division in preparing question- 
naires for investigations sponsored by the Na- 
tional Education Association and partly upon 
the advice of previous writers on this subject.’ 
Before listing specific suggestions, attention 
will be given to certain preliminary conditions 
which the investigator should settle before he 
begins to prepare his blank. 


Preliminary Considerations 


The topic investigated should be a worthy 
one. It is extremely difficult to set up objective 
criteria for judging the worth of a proposed 
subject of study. Yet it is true that some ques- 
tionnaires deal with educational questions 
which the great mass of competent critics 
would classify as significant, while others waste 
energy collecting data on matters of little im- 
portance or, as has been said, marshal battalions 
of data to prove something which everybody 
knew already and which makes little difference 
anyway. The advice of one or more experi- 
enced investigators is perhaps the best guide as 
to the worth of a subject proposed for ques- 
tionnaire study. If several experts agree that it 
is a topic which can best be studied by this 
method, then there is some warrant for pro- 
ceeding. 

The foregoing is particularly pertinent for 
the inexperienced investigator such as the typ- 
ical graduate student. It will be less helpful to 
the practical school executive confronted by 
the necessity, real or supposed, of quickly as- 
sembling data to shed light on some moot ques- 
tion of policy being considered by his school 
board. For example, it may appear a trivial 
matter to some people whether it is the practice 
to divide a teacher’s annual salary into nine, 
ten, twelve, or twenty-four payments. But the 
school administrator sometimes is unexpectedly 
confronted by a situation where he feels he 





1 See the bibliography, pages 47-9. 
2 See pages 44-7 for sources of information. 


must have facts and opinions on such a poin 
and in a hurry. What is he to do? Perh: 
the end circulate a questionnaire. But | 
deciding to do this it would be well +: 
these questions : 

1. Would a special delivery letter o: 
gram to some recognized agency of educatic; 
information ? bring the material desired ? 

2. Is it not likely that the flare-up which 
prompts the questionnaire will have passed 
fore the information needed can be assemb|ed ? 

3. Will not a little hard thinking on the 
subject by the superintendent and members of 
his staff be as safe a guide as the median of 4 
series of discordant practices, or the average of 
off-hand opinions, which are frequently what 
one gets in reply to a questionnaire on trivial 
topics? 

4. Will not a few hours’ search in the loca! 
school or public library be more productive 
than the same time spent in issuing and tabvu- 
lating a hastily prepared questionnaire ? 

If the answer is no to each of the foregoing 
questions then perhaps it is justifiable to dis- 
patch a questionnaire for the information de- 
sired. But the burden of proof is on the in- 
vestigator who takes the time of busy people 
to fill out a blank assembling data on ephemera! 
issues of local interest. 

Adequate sponsorship should be secured 
Discriminating respondents consider the spon- 
sorship carried by a questionnaire to be an im- 
portant criterion in deciding whether toanswe: 
it or not. If from a recognized agency, usuall; 
jealous of a good reputation, it is likely to have 
been carefully prepared. These are among the 
offices whose sponsorship carries a measure 
of guarantee that the questionnaire is worth 
answering : 

1. Office of superintendent of schools par- 
ticularly when his staff includes a person qual- 
ified to conduct educational research. 

2. Schools or departments of education func- 
tioning through a research bureau or a pro- 
fessor who personally sponsors the inquiry. 

3. Governmental agencies concerned with 
education, state and national. 
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4 State and national education associations 
with facilities for expert guidance in research. 
The majority of questionnaires sponsored by 
wencies comparable to the illustrations above 
are carefully planned, and will directly or in- 
directly render some return to those who an- 


swer them. 

The questionnaires most local 
teachers’ Officers of 
these organizations would do well to secure 
the aid of some qualified representative of an 
agency such as those listed above before sending 
out questionnaires. 

Graduate students of 
thesis material should secure the advice of one 
or more qualified investigators such as their 
professors or the assistance and endorsement 
of a recognized agency of before 
launching a questionnaire. 

Agencies with facilities sponsorship 
should take this responsibility seriously. The 
questionnaire should not only carry the name 
of the sponsoring organization, but that of a 
particular representative, qualified to appraise 
investigations. 

Most unnecessary and poorly prepared ques- 
tionnaires can be kept from circulation if ques- 
tionnaire-makers will seek expert advice on, 
and official endorsement of, their inquiry 
blanks, and if respondents return unanswered 
most of the questionnaires which do not carry 
these passports. 

The investigator should know the field—It 
has been said that he who takes little knowl- 
edge into a strange land carries back little 
knowledge. Likewise, he who knows little about 
a subject will be able to gain little through a 
questionnaire. In order to ask intelligent ques- 
tions in a field of study it is necessary first to 
become a student of the field. To construct a 
questionnaire inquiring about the organization 
of school systems, which assumes the existence 
of no other type of school organization than 
the 8-4-4 plan, would be folly. Yet inquiries 
making such assumptions have been circulated. 

The literature and research bearing upon 
the proposed investigation should be carefully 
studied. One is repeatedly surprised at the 


issued by 
organizations are poor. 


education seeking 


research 


for 


amount of material which is already available 


on nearly every educational question. It may 
not be the last word, and may merely testify to 
the need for more study on the same topic, but 
it deserves consideration. If it does not receive 
it, the principal outcome of the questionnaire 
may be to advertise the ignorance of the in- 
vestigator to those who know the field. Before 
drafting a questionnaire, therefore, prepare as 
complete a bibliography * on the topic as is pos- 
sible, and evaluate the references cited therein. 

The scope of the problem should be carefully 
delimited—The_ student of 
quently has the experience of beginning to 
study a subject which at first appears to be a 
unit question, only to find that it soon divides 
and ramifies into a whole series of issues, 
which must be dealt with one at a time. This 
fact justifies the advice: clearly define the prob- 
lem and scope of your study before you draft 
the questionnaire. Study the topic being in- 
vestigated until its various strands are sepa- 
rated, and a deliberate choice is made of those 
which are to be the subject of inquiry. 

Observance of this principle usually 


education tre- 


calls 
for a clear-cut decision as to whether the ques- 
tionnaire is to have wide scope and eschew de 
tails, or cover a limited field and delve deeply. 
Below are some of the questions of a successful 
questionnaire * which sought to touch lightly, 
but to range widely over the field of teacher 
personnel : 


Married Women as Teachers 


Has your board of education a rule o1 
against the appointment of new teachers who are 
married women? 

pf Pe eer ree 

(Indicate answer by check mark.) 

Does your school board require women teachers 
who marry to give up their positions at once? 
ee ee ee MO ce BE seks ous At the end of the 
CONGUE FOEET "Tbsicciss pecs xs 

(Indicate answer 


policy 


by check m ask.) 


Written Examinations 


Does your school system conduct written examin- 
ations in any form in connection with employing 
new teachers? 


! A librarian will be able to cite the guides to educational literature most helpful in preparing a bibliography. Also consult 
such references as numbers 18 and 24 in the bibliography i in this Bulletin. Reasonably recent and complete bibliographies on many 
topics may be obtained from sources of information given on pages 44- 7. 


age 251, 


* See p om Affecting Teacher Personnel,” 
1928, Vol. Vi. No. 4 


Research Bulletin of the National Education Association, September, 
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(Indicate answer by check mark.) 

In connection with promoting employees within 
the service? Yes............ 
(Indicate answer by check mark.) 


Sick Leave 


What part of the regular salary is paid to teach- 
ers taking sick leave? (Check below.) 
Ere 
ee eee eee 
Full salary, less pay of substitute. .... 
Some other fraction of salary.... 
Ser 
If sick leave with some salary is allowed: 


a. How many days are allowed in one 


b. Is the allowance cumulative, i. e., may the 
days not used during one year be added to 
those of later years? 

Mis sev auevance 
(Indicate answer by check mark.) 

c. During how many years may sick leave allow- 

ance accumulate? 


eee een en nnee 


(Indicate no. years.) 


‘The-questionnaire method is at its best when 
it employs questions such as the foregoing. It is 
less satisfactory in securing details of operation. 
Personal interview or individual correspond- 
ence is often the only reliable method of getting 
such information. It is hard to go wrong in an- 
swering the preceding questions. But when one 
attempts to develop one of these into a series of 
detailed questions, he should anticipate a re- 
duction in the reliability of replies. 

The wide-ranging as opposed to the deep- 
delving questionnaire is particularly valuable 
in uncovering pay-ore territory for future in- 
tensive investigation. For example, the ques- 
tions above on examinations would reveal the 
cities worth investigating if a study were pro- 
posed as to the use of written examinations in 
selecting and promoting teachers. Several 
studies in the teacher-personnel field have made 
their take-off from data revealed by the ques- 
tionnaire partly reproduced above. 

Whether one’s inquiry is to deal intensively 








or extensively with a field will depend oy +, 
object and scope of the study. If this has beep 
clearly thought out, one will know when , 
use the extensive as opposed to the intensiy. 
type of question. 

Data should not be collected by questi, 
naire if otherwise obtainable—lf the invest; 
gator will take thought on this point before 
preparing a questionnaire form, he may dis 
cover other means for getting the materia! he 
needs. It may be found that intensive, persona! 
study of a few situations will produce more 
valuable results than a questionnaire sent 
broadcast. School records and reports, govern 
mental reports, and the publications of educa- 
tional foundations and associations contain 4 
wealth of raw data of value to the investigator, 

One who has carefully studied published ma- 
terial in the field of his inquiry is not likely to 
ask for information already available. ‘Vhe {o!- 
lowing statement is pertinent with regard to 
the unnecessary use of questionnaires : 


In the first place let us remember that the ques- 
tionnaire is often a lazy man’s method of getting al! 
leged information that could be much better ob- 
tained in some other way. It is easy to sit down in 
the quiet of one’s study and make out questions for 
someone else to answer. But such a method is very 
wasteful of the time and energy of the one to 
whom it is sent and is frequently wasteful of the 
time of the maker, for these questionnaires not in- 
frequently find their way into the waste-paper 
basket, and many of them should. One has no right 
to take the time of someone else in answering ques 
tions when he himself can find the answer else- 
where. Nor, on the whole, can he consistently ask 
someone to spend his valuable time in work that 
will not presumably prove of value to him or his 
school. After he has decided upon the data he 
wishes, he should search through all available ma- 
terial to see whether these data cannot be obtained 
through printed reports. If they can and he does 
not have the reports, he should send for them and 
get the data for himself. If the data given in the 
reports are not in the exact form in which he wishes 
them, he should change his own plan to fit the form 
used, where this is possible.* 


The follow-up should be anticipated—lt \: 
usually desirable to send at least one follow-up 


1 The following is a good example of a study which, although based on but a small number of cases, exceeds the value of min) 
broadcast questionnaire studies: Rufi, John. ‘The Smal! High School."’ Contributions to Education, No. 236, Bureau of Publications 
Teachers College, Columbia University, New York City. 1926. 145 p. 


* Jones, Arthur 
Fog Suggestions for 
.s 


.. Chairman, National Committee on Research in Secondary Education, Am Outline of Methods of Resear 
igh School Principals and Teachers. U. S. Bureau of Education, Department of the Interior, Bulletin, 19/6 
Government Printing Office, Washington, D. C. 1927. 31 p. 
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letter a short time after a questionnaire has 
wer) mailed. The vigor and number of follow- 
; will depend upon the questionnaire. If fail- 


ul 
to respond is an outgrowth of some factor 


ure 


of selection, then a follow-up campaign is nec- 
essary. But the danger of a selected response 
does not always hold, as Koos points out: 


(his danger by no means always obtains. It is 
important for the investigator to consider the likeli- 
hood and influence of selection and, if a practically 
full count seems essential, take the necessary steps, 
such as repeated inquiry, to secure it.’ 


Toops’* study, in which a 100 per cent re- 
turn on a questionnaire was obtained through 
the use of six follow-up letters, supports Koos’ 
contention that the danger of a selective re- 
sponse is not present in all questionnaire in- 
vestigations. The essential conclusion of Toops’ 
study is that additional replies brought in by 
the follow-up letters did not essentially change 
his conclusions, 

Lindsay's * investigation suggests that when 
follow-up letters are to be used they should 
begin to go out within two weeks after the 
questionnaire has been mailed, rather than 
waiting longer, and until there is a slackening 
in the receipt of replies. 

The cost of providing and mailing the fol- 
low-up letters should be anticipated as a part 
of the total cost of the investigation and before 
the decision to make the study has been made. 

Plans for tabulating should be formulated 
early *—-Too often investigators have little 
idea of the time required to tabulate replies. 
The authors of a number of the questionnaires 
followed up by the Research Division reported 
that no use was made of the replies because of 
lack of facilities for tabulating. Others re- 
ported the work partly completed but at a 
standstill due to the lack of facilities for com- 
pleting the job. Such an embarrassment can be 
avoided if the investigator will have from five 
to ten questionnaires filled out in the tryout 
suggested on page 22, and will keep an accurate 
count on the time required to tabulate them. 


' Koos, Leonard V., The Questionnaire in Education, p. 167. 


An estimate can then be made of the total time 
required to handle the expected number of 
replies. If it is more than one is equipped to 
furnish with reasonable promptness, then the 
questionnaire should be shortened and the aum- 
ber sent out reduced. 

The method of disseminating results should 
be settled—This involves typing, and duplica- 
tion in some form, printed or otherwise. The 
cost of duplicating the results of an investiga- 
tion in quantity sufficient so that there can be 
a reasonable dissemination of results will be 
considerable, particularly if it involves tabular 
material. The probable number of tables, their 
size and character, and the amount of running 
text should be estimated ahead, and an esti- 
mate of the price for mimeographing or print- 
ing obtained. 

Finally, there is the problem of mailing the 
study, or at least of notifying respondents of 
its completion and availability. This should be 
an integral part of every questionnaire investi- 
gation. At the very least, the questionnaire 
should contain a question such as “Do you de- 
sire a copy of the tabulated replies to this 
questionnaire?” and all who respond affirma- 
tively should be taken care of. Preferably, the 
questionnaire should make a definite promise 
that a copy of the results will be mailed to all 
respondents, or at least that respondents will 
be promptly notified when results are available. 
The latter practice is acceptable when the study 
appears in a magazine or as a separate publica- 
tion, the cost of which is too great to permit 
free distribution. 

When the material is to be used by a gradu- 
ate student in a thesis it is enough, and usually 
preferable, that a brief summary of the results 
be mailed to the respondents. But enough copies 
of the complete study should be available so 
that a copy may be borrowed by an interested 
respondent. Few things have done more to 
bring the questionnaire method into ill repute 
than the practice of some universities ° of per- 
mitting graduate students to bury a few type- 
written copies of the results of questionnaire 


* Toops, H, A. “Returns from Follow-up Letters to Questionnaires."’ Journal of Applied Psychology. 10: 92-101; March, 1926. 
* Lindsay, E. E. ‘Questionnaires and Follow-up Letters.” Pedagogical Seminary 28: 303-7; September, 1921. 


* Also see pages 21, 25, and 36. 


§ The Research Division in seeking to secure copies of unpublished graduate dissertations for the use of important committees 
of the National Education Association has, in some instances, found it a time-consuming poegen to borrow these investigations 


from some college libraries. The question is raised: If a study is of sufficient vakie to justi 


y the granting of an advanced degree 


should it not be a requirement that it be printed in full, if for the doctorate, and at least in summary form if for the masters degree? 
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studies, some of which are of real significance, 
in the university library. 

If the investigator is convinced that the in- 
formation he wants will shed light on a worth- 
while question, that his questionnaire is well 
sponsored, that he has a sufficient mastery of 
the subject to prepare an intelligent question- 
naire, that he has carefully delimited the scope 
of his investigation and therefore knows just 
what he wants, that he can get the desired data 
only through a questionnaire, that he is in a 
position to handle the replies with reasonable 
promptness, and to render respondents a proper 
return for the time which they will donate, 
then only is there justification for actually 
drafting the blanks. 


Preparing and Sending the Question- 
naire 


The following guides should be suggestive. 
They are not invariable principles but can be 
observed with profit in most cases. 

The questionnaire should preferably deal 
with matters of fact—This method is often at 
its worst when it attempts to discover truth by 
collecting subjective opinions or judgments. 
It is extremely difficult to anticipate all the 
implications and qualifications which will affect 
the respondent as he seeks to reply to a subjec- 
tive question. The following may appear to be 
a simple question: 


Do you believe school systems should adopt the 
platoon school type of organization? 
Yes No 


But when one comes to answer it a swarm of 
considerations arise to befuddle the respond- 
ent. Just what is meant by a platoon school ? 
Is this form of organization equally effective in 
all grades? Would it not be better to experi- 
ment first in a few schools? Are not the school 
plants of many cities such that it would be im- 
possible to install the platoon plan, except at 
its worst? At about this time the respondent 
either flips a coin to determine his answer, or 
throws the questionnaire into the waste-basket, 
The prejudice felt against the opinion-seeking 








questionnaire is stated by one school exe 
as follows: 


The true questionnaire devotee never asks 
information desk at the North Station, “Wh: 
the next train leave Boston for Springfield 
seeks the opinion of all experienced travelers 
in the seats of the waiting room and, with hi 
rule busily working, he hurries to the train 


The following are some of the reasons \ 
suggest the desirability of reducing the nu 
of questions calling for subjective replic 
minimum : 


1. Questionnaires seeking opinions get a relatiy: 
poor percent of reply.’ 

2. The hastily expressed opinions of a large nun 
ber of judges of varying merit are probably of 
less value than the mature judgment of one o; 
two experts who have the question clearly j; 
mind, and who are able to qualify their answer, 
appropriately. 

3. Answers to questions of opinion if thoughtfu!|) 
given are time consuming, and cannot be del 
gated to a clerk. 


The foregoing should not be interpreted as 
a blanket condemnation of a certain valuable 
type of questionnaire investigation in which 
the attempt is made to study opinions as such 
without regard to their innate value. In such 
cases opinions. become a species of fact worth 
of study in and for themselves because of the 
light which they may throw on problems. ||lus- 
trations of studies of this type are cited be- 
low.® 

Both Koos* and Kelley defend the ques- 
tionnaire as an instrument for determining 
consensus of opinion. After pointing out tl 
difficulty of securing exact measures of human 
opinion, Kelley says: 


“The instrument for attacking this problem is the 
questionnaire. Is this not a superb anticlimax’ 
Here I have described the most difficult field of re- 
search, and that possibly the most pregnant for 
human welfare, and propose for its handling th 
weakest device that has ever obtruded its prying 
and at times discourteous self into the respectab 
field of science. Though I must admit all of this, 
I would ask how in the realm of mental values is 
any approach conceivable except that which ascer- 


- } Seay ny E. W. “The Plenary Inspiration of the Dotted Lire.’’ Educational Review 69: 1-4; January, 1925. 


See Squses given on page 26. 


‘Jones Verao 


My ey t Among Teachers as to Right and Wrong." Teachers College Record 31: 24-36; October. 192° 
id, William Hall. ht Citizens Know About Their Schools. Contributions to Education, No. 279. Bureau of Publicatio: 


Teachers College, Columbia University, New York City. 1927. 86 p. 
Leonard 


* Koos, 


V. The Questionnaire in Education, page 147. 
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Experts can perhaps use the questionnaire 
effectively in securing a valuable consensus of 
1uman opinion on certain topics. ‘The begin- 
ner in the field should reduce the number of 
questions asking for subjective opinion to a 
minimum. 

Especial care should be used in selecting a 
croup of respondents who are competent to 
express an opinion on the subject of investiga- 
tion. The number of replies is less significant 
than who replies—quality is of far greater 
importance than quantity. 

Ask only for data which respondents can 
and will give—Koos emphasizes this princi- 
ple under the criteria of ability and willing- 
ness.2 When dealing with matters of fact, the 
questions should ask for information which 
the respondent may reasonably be expected to 
have available. When dealing with matters of 
opinion or judgment, the competence of the 
respondents to give the desired judgments 
should be considered. 

Questions dealing with matters of fact may 
ask for information in the personal knowledge 
of the respondent, or for information reason- 
ably available from school records. Care should 
be taken to keep to a minimum the number 
of questions which will require extensive 
searching of school records. The extent to 
which an official will go in searching out in- 
formation not readily available will depend on 
the apparent value of the investigation. 

Care should be taken to include in the ques- 
tionnaire blank only questions which can be 
answered by the person addressed or by his im- 
mediate staff. For this reason it is usually un- 
wise to include in the same blank a series of 
questions which must be circulated among a 
number of people, for example, the high school 
principal, the general supervisor, the special 
subject supervisor, or other specific school off- 
cials. The answering of such an inquiry entails 
considerable correspondence to round up re- 
plies from the several sources of information 
and therefore places a burden on someone. For 





is likely to be low, questionnaires which re 
quire replies from several different persons 
should be eschewed whenever possible. When 
they are necessary, the cooperation of the super- 
intendent of schools should be enlisted in the 
project before the questionnaire is sent to a 
city. 

Questions are sometimes asked which it is 
dificult for anyone to answer except by a wild 
guess. The following illustrations are from 
actual questionnaires: 
been 


When Cumulative Sick Leave has well 


established, do you consider the annual amount for 
sickness allowed by the Board would be materially 
greater than before the time of making Sick Bene 
fits Cumulative? 

If you have in use the single salary plan, what 
was the effect of its introduction on the salaries of 
high school teachers? 

Is your janitor interested in the welfare of the 
children? Is the manner of his voice pleasing? 


The inclusion of such questions shakes the 
confidence of the respondents in the competence 
of the investigator. Replies to such questions 
are of little value. 

Then there is the question which indiscrimi- 
nately asks ages, or other personal questions 
which many dislike to give. Sometimes an hon- 
est reply might later embarrass the respondent, 
for example: 

Does the school board ignore the superintendent 
in many instances and thus assume authority right}, 
belonging to the superintendent, yet holding the 


superintendent responsible for the general success 
of the schools? 


The willingness of the respondent to reply 
needs to be anticipated. Questions which call 
for information of a highly personal nature, 
which call for answers reflecting on others, 
and which prematurely commit one on highly 
controversial issues are likely to reduce the 
percent of reply. When it is necessary to in- 
clude such questions a promise to report re- 
sults in such form that they can not be traced 
to individuals, or a waiver of signature, may 
increase the willingness of the respondent to 
give complete and accurate replies.° 

The questionnaire should be as short as pos- 





this reason, and because the percent of replies sib/e—Short questionnaires tend to obtain a 
' Kelley, Truman Lee. Scientific Method. p. 39. 
* Koos, op. cit., p. 99. 
* Koos, op. cit., p. 99. 
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higher percent of return than long ones. ‘The 
present investigation included twenty ques- 
tionnaires containing five items or less, which 
received a median return of 78.5 percent, as 
compared to a 69.0 percent return on the 136 
investigated. Table 9, page 34 shows that, in 
general, the percent of return decreases as the 
length increases. 

An effort should be made to eliminate every 
question not essential to the problem. Many 
questionnaires contain one or two queries the 
replies to which can not be utilized. The time 
of everyone concerned would have been saved 
by the omission of such questions. 

On the other hand, the few replies to a long 
and detailed questionnaire may be more valu- 
able to the investigator than a nearly complete 
response to a shorter questionnaire on the same 
topic. The investigator should weigh the pos- 
sible advantages of a high percent of replies to 
a short questionnaire, as compared with a large 
amount of information from the few generous 
respondents who will answer a long one. 

In certain cases the long questionnaire, which 
has been carefully prepared, which is adequately 
sponsored, and which deals with a subject of 
general interest, receives a high percent of re- 
turn. For example, the longest and probably 
the most difficult questionnaire in the 267 ob- 
tained for this study received a return of 88 
percent. The blank related to the cost of in- 
structional supplies, and their distribution to 
the various grades in the school system. It was 
16 pages long, and called for 1,833 items of 
financial statistics. It had been preceded by a 
letter to superintendents of schools, describing 
briefly the plans for the study, and inviting 
them to cooperate. The superintendents of 125 
cities expressed a willingness to cooperate, and 
110 of them furnished the information re- 
quested, 

Questions should be simply and clearly 
worded—No respondent should be expected to 
puzzle out a questionnaire. One should be able 
to make his wants known in simple language. 
If he cannot give a brief and clear explanation 
of the plan of his inquiry, it is probably not 
suited to the questionnaire method, and he 
should either simplify his plan or change his 
method of collecting data. 





The simplest form of inquiry, and the | 
most easily answered, consists of a series 
complete questions, each of which can st 
alone, independent of context. For the s| 
questionnaire, this is the most desirable {, 

When an inquiry is very long and deta 
howeyer, it may be simplified by casting it i; 

a tabular form with directions to the respon 
ents as to the method of answering. In | 
paring a tabular form for recording statisti: 
data, great care is required in the wording of 
column headings and definition of items i; 
cluded. The blanks used by the U. S. Offic: 
Education * for the collection of statistics on 
school personnel and finances illustrate th 
working out of this principle in a practi 
situation. 

The tabular form may also be used to sim 
plify the long questionnaire dealing with non- 
statistical data. For example, a questionnaire 
dealing with functions in health education may 
list a series of possible activities carried on in 
this work, preceded by a request that the re- 
spondent check those functions performed in 
his local school system. Or it may be somewhat 
more detailed, providing separate columns 
headed “classroom teacher,” “school nurse, 
and “school doctor,” with directions to check 
the functions performed by each. 

But the investigator must guard against 
too great complexity. To carry further the 
example of the questionnaire on health func- 
tions, he might ask the respondent to use one 
check when the function is performed fre- 
quently, and two checks when performed in- 
frequently, and to indicate in another column 
by letters “A,” “B,” and “C,” the relative im- 
portance of having the function performed at 
all. Unrestrained, he may even add additional! 
columns to be filled. in, headed “Grades In- 
cluded,” “Number of Pupils Reached,” “Yea: 
When Function Was First Performed,’ and 
“Desirable Outcomes of the Function,” the 
latter column provided with a numbered code 
of possible answers, and instructions to ente! 
the code number instead of the actual answer. 

Questionnaires of this complicated type fre- 
quently ask for data of value, but they defeat 
their own purpose through their bewildering 
plans for getting responses. The investigato: 


' See page 289 ff. of “School Records and Reports,”’ Research Bulletin of the National Education Asscciation, Vol. V, No. 5. 
[ 20] 





I 


le aaa, seta 














scl 


ley 


Ca: 


sid 


Put 


Psy 








Erney 


ell Dee 








| 
| 


cannot expect his own interest to be equalled 
by the respondent to the extent of puzzling out 
an involved and intricate scheme for recording 
facts. 

The necessity for clear wording applies not 
only to the general plan of the questionnaire, 
but to the separate questions within it. Each 
question should be so clearly worded that all 
respondents will interpret it the same way. It 
is only by the most painstaking efforts, pre- 
liminary trials and revisions, that questions can 
be prepared which are entirely clear, and which 
will mean the same to all respondents. Such an 
ideal is hard to achieve, but the value of the 
results of the study will depend largely on this 
factor. 

That the form in which a question is asked 
influences the reply received has been pointed 
out by investigators dealing with the question 
in various situations.’ The order of a question 
in a series also affects the answers which will 
be given.” The significance of these facts should 
be understood by those who would use the 
questionnaire effectively. 


— 


The questionnaire should require a minimum 
of writing—Every effort should be made to re- 
duce the writing necessary to answer the ques- 
tionnaire. It should be worded so that the an- 
swers may be expressed by yes or no, check 
marks, underlining or encircling, by figures, or 
by short answers of a few words. 

The investigator wants to be sure, however, 
that his questions are suited to this type of re- 
sponse. In wording a question to be answered 
yes or no, one should see that it can be an- 
swered with an unqualified affirmative or nega- 
tive. The following are examples of yes or no 
questions to which few school officials could 
give a categorical answer : 


Is there more than one clerical assistant in each 
school ? 

Is the cost of building foundations below street 
level, underground pipes, excavations, and side- 
walks, included in the total amount of insurance 
carried? 

Is your department so organized that you con- 
sider the interpretation of the course of study to 





your staff members more important than the con- 
struction of the curriculum? 

If you have kindergartens, do they accommodate 
ill eligible pupils? 


When listing a series of answers for check- 
ing, underlining, or encircling, care should be 
taken to cover all the possible cases, or to pro- 
vide space for “Other Response.” In many cases, 
it will be desirable to ask a direct question to be 
answered with a few words. When such a short- 
answer question is so clear that its meaning 
is the same to all respondents, the replies are 
comparable. It is more likely to bring to light 
unusual practices than questions which list a 
series of alternative answers. 

Care must be taken, however, to limit the 
scope of the short answer question so that its 
replies can be tabulated and summarized. ‘The 
question which invites the respondent to write 
an essay produces replies which cannot be 
tabulated. Rather than essay questions, it is 
better to ask short answer questions, and to 
provide a space for “Remarks” where addi- 
tional details will be recorded by those willing 
to give them. 

Responses should lend themselves to tabula- 
tion—Rugg has well summed up the advice 
which is pertinent on this point: 

The answers to “general” questions seldom can 
be tabulated and definitely interpreted. It is a safe 
rule to follow that data which do not lend them- 
selves to tabulation and statistical treatment are of 
negligible value to the investigator. That is, an- 
swers should be definite and susceptible of tabular 
classification and this should be a controlling cri- 
terion in the planning of questions.* 


Emphasizing the thought still further Rugg 
suggests that the investigator work out the 
tabular forms for entering the replies to his 
questionnaire at the same time that he prepares 


his blank: 


This means the planning of the specific headings 
of the tables to be compiled, and will require defi- 
nite decisions concerning the arrangement of ques- 
tions and the probable types of returns to secure. 
Preparation of the tables will lead to a clearcut, 
logical arrangement of questions, so put together 
as to facilitate a clear presentation and discussion 


1 Weidemann, Charles Conrad. How to Construct the True-false Examination. Contributions to Education, No. 225. Bureau of 
Publications, Teachers College, Columbia University, New York City. 1926. p. 41-67. 
Muscio, Bernard. “The Influence of the Form of a Question.” British Journal of Psychology 8: 351-89; September, 1916. 


* Mathews, C. O. 
Psychology 128: 34; February, 1929. 


The Effect of the Order of Printed Response Words on an Interest Questionnaire.” Journal of Educational 


* Rugg, Harold. Statistical Methods Applied to Education. Houghton, Mifflin Company, Boston, 1917. p. 47. 
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in the report. A little time spent at this stage of the 
work will aid much in the later organization of the 
completed discussion.* 


Each question should be so worded that it 
can be answered by a single statement. Ques- 
tions are frequently asked which require two 
answers, such as the following: 

Are you contemplating a change in history text- 
books, if so which one are you considering? 

Do you study occupations as a group or are the 
boys and girls receiving separate instruction? 

Has any legal action directly applying to pri- 
mary grades been taken and what? 

Frequently the respondent will read only 
half of such a double question and answer yes 
or no to the despair of the tabulator. 


A careful examination of individual ques- 
tions with this principle in mind will often re- 
veal some of the replies which it will be im- 
possible to tabulate, or which will consume 
much of the tabulator’s time and give uncer- 
tain results at best. Such questions had best be 
eliminated or recast. 

The questionnaire should be given a prelim- 
inary tryout—The extent of the preliminary 
trial will depend on conditions. It should be 
remembered, however, that the dependability 
of the answers may be strengthened materially 
by thoroughly trying out the form. One easy 
test, which should be made of even the simplest 
questionnaire, is to try it out locally. Before 
duplicating the form, the investigator should 
have typed copies of the questionnaire answered 
for his own local school system. If a few copies 
are sent out to nearby cities, the replies will 
almost inevitably suggest needed modifications 
of the form. Questionnaires goiny to other 
groups may similarly be tried out first on a 
small sampling of those to be circularized. 

The National Committee on Research in 
Secondary Education suggests that before 
sending out a questionnaire it should go 
through the following stages: 

(a) Very careful formulation by the author and 

arrangement in the form to be used. 

(b) Submission to some expert for advice and 

correction. 


(c) Tryouts on teachers or others not primarily 
concerned—disinterested persons. 





1 Ibid, p. 50. 


for High School Principals and Teachers. U.S. 
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(d) A tryout of the revised questionnaire 
group as nearly like the ones to whom it is 
be sent as possible. These tryouts will! of; 
show the inaccuracies of statement, ¢! 
equivocal questions, and other undesira! 
features that can be corrected before 
questionnaire is actually sent out for replics 


If all questionnaires were to pass throu; 
the foregoing stages, their authors would not 
only save their own time and that of their : 
spondents, but would get far more valuable 
products for their labor. 

A questionnaire should meet certain stand 
ards of mechanical form—A_ questionnair: 
which is poorly arranged and carelessly du 
plicated is likely to raise doubts in the minds 
of the recipients as to the ability of the in 
vestigator to follow a study through to a suc 
cessful conclusion. The response will therefore 
be reduced. A neat, well-arranged question 
naire encourages an accurate, businesslike reply. 
These are among some of the standards of 
mechanical form which should be observed: 

a. The questionnaire should be neatly re- 
produced. Typing, if well done, is acceptable if 
only a small number is being circularized. 
Mimeographing, multigraphing, and electric 
typewriting are duplicating processes which 
produce copies resembling typewriting, and are 
satisfactory for short questionnaires going to a 
limited number. Printing is the most desir- 
able method of reproducing questionnaires, and 
tends to get a better reply. It is especially ad- 
vantageous when the questionnaire is long, in- 
volves tabular and columnar arrangements, and 
is going to a hundred people or more. 

b. The form should contain spaces in which 
to enter the name, offical position, and full ad- 
dress of the respondent. Allow ample space. |t 
is poor policy to irritate the respondent at the 
outset by allowing only half enough room for 
his name and address. Even with such space 
provided some will forget to use it. It is de- 
sirable to know who has filled out a question- 
naire in case later correspondence is necessar) 
and for other reasons. If envelopes are pinned 
to questionnaires as they are opened, the post- 
mark will aid in identifying them if signatures 
are omitted. Earlier in this section is discussed 


* Jones, Arthur ii Chairman of National Committee on Research in Secondary Education. An Oudline of Methods of Resea 
Department of Interior, Bureau of Education. Bulletin, 1926, \ 
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the circumstances under which it may be de- 
sirable to waive the signature and other iden- 


tification. 

c. The questionnaire itself should contain 
he name and address of the investigator. 
Otherwise, the transmitting envelope is likely 
to be thrown away, the letter of transmittal 
lost, and the questionnaire left homeless. Some- 
times they are sent to the wrong address and 
cannot be forwarded as there is nothing to in- 
dicate who sent them out. 

d. Ample space should be left for the an- 
swering of each question. A crowded question- 
naire allows no space for the answer longer 
than average, and encourages incomplete and 
slovenly replies. Experience has shown that a 
two-page questionnaire, generously spaced for 
replies, gets a better reception and response 
than if it is crowded into a single page. The 
tryout of the form will suggest the amount of 
space necessary for the answer to each ques- 
tion. It is desirable to have a line or two after 
certain short-answer questions, and at the close 
of the questionnaire for “remarks.” Unusual 
practices, significant qualifications and other 
material often of great importance to the in- 
vestigator, will be written in if space is pro- 
vided. 

The purpose of the investigation should be 
stated—More than one-third of the question- 
naires studied failed to state the intended use 
of the results. Recipients are likely to be sus- 
picious of investigations which have no visible 
purpose. It is not enough merely to be “inter- 
ested in the problem of sabbatical leave,” or to 
state ‘‘l am making a study of child accounting 
systems.” Rather state ‘‘we are in need of tliese 
facts to help us establish a sound policy in the 
payment of teachers during sickness,” or “I 
shall use this in a dissertation at Blank Univer- 
sity, which is being written under the super- 
vision of Professor Doe. A summary of the re- 
sults will be mailed promptly to all respond- 
ents.” Frankness on this point gives assurance 
that the material requested is to serve a useful 
purpose and begets cooperation. 

The statement of purpose may be given in a 
letter of transmittal or in an introductory 
paragraph on the questionnaire. It should be 
brief, but adequate, and should not be omitted. 








Buckingham would go even farther, and his 
advice is particularly pertinent if the question- 
naire is long and involves the assembling of 
material not immediately available. He states: 

The receiver of the material should be taken 
fully into the investigator’s confidence and should 
form his own judgment of the value of the under- 
taking. It is generally wise to write in advance to 
the intended recipients of the questionnaire and to 
acquaint them with the project. Ask them if they 
believe it to be worth while and if they will fill 
out the necessary blanks. Otherwise what right have 
we to “fire” a battery of questions at a group of 
busy people? To do so is both inartistic and dis- 
courteous. There is a brusque sort of effrontery in 
printing all the hard questions one can frame on a 
given subject and thrusting them without so much 
as “by your leave,” before the face of a distant 
and therefore defenseless victim. The stamped self- 
addressed envelope adds the last excruciating touch. 

Investigators could raise considerably the esti- 
mation in which the blanks are held and the will- 
ingness of people to make them out if they would 
add to their technical skill in constructing forms 
an element of restraint and a modicum of courtesy.’ 

Other desirable practices. There are certain 
other customs which are often desirable under 
the principle of courtesy, if for no other reason. 
They follow: 

a. A separate letter of transmittal is usually 
desirable. It offers an opportunity to explain 
the purpose and extent of the investigation and 
to give other pertinent information. It should 
be brief and to the point. Such a letter may be 
addressed in general terms, for example, ‘““To 
the Superintendent of Schools” or “To the 
Teacher of Mathematics.” Still better, the 
name and address of the individual receiving 
the questionnaire may be typewritten on a mul- 
tigraphed form, Individual letters to each re- 
spondent are desirable in some cases, partic- 
ularly when the number addressed is less than 
fifty and the cost is therefore little more than a 
form letter. If any large number of question- 
naires is being sent out the cost will be least 
for a general form letter, and most for per- 
sonal individual letters. In some cases, par- 
ticularly when the questionnaire is very short, 
the letter of transmittal may be a part of the 
questionnaire itself, and the cost thereby re- 
duced. 

b. It is usually desirable to send two copies 
of the inquiry form to each respondent. The in- 


' Buckingham, B. R. “The Questionnaire.” Journal of Educational Research 14: 54-8; June, 1926, 
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formation requested is frequently of value to 
the respondent. The extra copy allows him to 
record it in proper form, and to have an exact 
record of his report in case any question later 
arises. ‘his courtesy is desired and sometimes 
demanded, particularly by offices of superin- 
tendents of schools. The cost of the extra copy 
is not great when the form is mimeographed or 
printed. On the first page of the blank should 
appear the note: “Two copies of this blank are 
sent, one to be returned to the address indi- 
cated, and one retained for your files.” 

Extra copies will also be needed if a follow- 
up is to be conducted in order to increase the 
percent of reply. In order to provide duplicate 
copies for respondents, to conduct the follow- 
up, and for other reasons which sometimes de- 
velop, it is usually desirable to print three times 
as many blanks as the number of persons to be 
circularized. The cost of securing an ample 
number is less if it is done at first, rather than 
reduplicating or reprinting the blank. 

c. It is usually desirable to enclose a return 
envelope. The size of the envelope should be 
determined by the size of the question blank. 
The envelope should usually be stamped. There 
are cases when this may not be necessary, par- 
ticularly if the questionnaire is going to well 
equipped offices, and the results of the study 
will be of obvious value to the respondent. 

Summary and conclusion—This_ chapter 
draws upon the experience of previous investi- 
gators and of the Research Division and sug- 
gests certain practical guides to questionnaire 
investigation. It is stated that previous to pre- 
paring a questionnaire one should be convinced 
that the information sought will shed light 





upon a worthwhile rather than a trivial iss), 
that the investigation is well sponsored, thar | 
has a sufficient mastery of the field to be st 
ied, that he has carefully delimited the scope 
his investigation, that the desired data can 
obtained only through questionnaires and ¢}).; 
he has facilities with which to handle the re; 
with reasonable promptness and to rende: 
spondents a proper return for the time the) 
will donate. Satisfied as to these essential pre 
liminaries the investigator may proceed to | 
drafting of his blank observing these criteyia 
of good questionnaire preparation: The ques 
tion blank should preferably deal with matters 
of fact, which respondents are able and willing 
to give, rather than with opinion or judgment. 
The number of questions should be reduced to 
a minimum by eliminating non-essential in 
quiries, be clearly and simply worded, and re- 
quire a minimum of writing by the respondent. 
The response to questions should lend them 
selves to tabulation. This and other desirable 
features can be best guaranteed by a prelim 
inary tryout of the blank. The questionnaire 
should meet certain standards of mechanical 
form such as legibility, neatness, proper spaces 
for responses, should be fully identified, and 
should be accompanied by a brief and frank 
statement of the purpose of the investigation 
It is desirable also to provide a letter of trans- 
mittal, extra copies of the blank, and a return 
envelope. These criteria of good questionnaire 
preparation should not be looked upon as in- 
variable principles but usually can be observed 
with profit particularly by inexperienced in- 
vestigators. 
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Reporting Questionnaire Results 


Many books and articles are in print which 
describe proper statistical and other procedures 
in handling educational data. One who would 
make effective use of facts assembled by ques- 
tionnaire should be familiar with these writ- 
ings. They apply quite as much to the handling 
of data from questionnaires, as to those from 
other methods of investigation. 

This chapter will not attempt to recapitulate 
the general principles which should guide one 
when he reaches the stage of presenting the re- 
sults of an investigation. Rather, at the risk of 
stating the obvious and of dealing with details 
too trivial to mention, it will make certain spe- 
cific suggestions, the observance of which would 
improve many studies based upon a question- 
naire. These suggestions should be of most help 
to the relatively inexperienced investigator 
with meager resources for securing expert ad- 
vice in reporting the results of his study. 

The report should be properly identified— 
Some of the questionnaire reports reaching the 
Research Division, particularly those in mimeo- 
graphed form, contain little to indicate where 
and by whom they were prepared. Every study 
should have a title, stating the subject of in- 
quiry as definitely as brevity will permit. The 
name, official position, address of the investi- 
gator, and the date of the investigation also 
should be given. These are essential even though 
the belief is that the report is purely of local 
interest. It is best not to be too modest in this 
regard. The investigation may come to the at- 
tention of outside people, even if only by acci- 
dent, and may be of interest to them. It will 
be of more value to the outside investigator if 
he has some means of identifying the origin, 
the age of the data, and the position of the in- 
vestigator. 

There should be an organized presentation. 
No rule of thumb can be laid down as to how 
one should proceed in presenting the results of 
a questionnaire investigation. Different studies 
will deserve varying treatments. An organiza- 
tion of one’s report under a series of headings 
such as the following will usually be desirable: 


a. Statement of purpose or problem of study. 





A description of the unsolved question on which 
the study sheds light. 

b. A brief description of how the data were 
collected, including an estimate of the relia- 
bility of the information secured, under such 
topics as: the number of questionnaires sent 
out, how those circularized were selected, the 
percent of reply, an estimate of the influence 
of the factor of selection if less than 100 per- 
cent replied, and anything else which an honest 
evaluation of the data should suggest. 

c. Presentation of data, including an ex- 
planation of what each table deals with, and 
what it seems to show. 

d. Statement of the conclusions based upon 
the data. This should not be confused with the 
preceding point. For example, it is one thing 
to state the fact that 60.7 percent of the cities 
in the United States do not appoint married 
women to new teaching positions, and it would 
be quite another thing to conclude from this 
fact that most superintendents of 
believe married women are poorer teachers than 
unmarried. In the first instance we have a fact 
which cannot be challenged, the latter is a con- 
clusion based on the fact, which may or may 
not be justified—probably the latter. 

e. Summary and conclusions. This should 
round up the study in a few succinct state- 
ments. The ease with which this takes shape is 
an excellent test of one’s clarity of thinking and 
writing up to this point. A well organized 
study is easily summarized, even by a stranger 
to the study. If it is hard for the investigator 
to tell the conclusion of the study, how can a 
reader expect to discover it? 


schools 


The foregoing headings should not be slav- 
ishly followed in writing up questionnaire re- 
ports. Variations and modifications will fre- 
quently be desirable. In a long investigation 
the foregoing headings may virtually reoccur 
over and over again as one table after another 
is presented, evaluated, and interpreted. Just 
how one organizes a study is less important 
than that it should be organized. 

Inexperienced investigators sometimes pre- 
sent data solely in a series of tabulations. This 
is inadequate even though the tables are well 


1 See such references as Nos. 1, 10, 14, 23, 24 and 30 in the bibliography. 
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set up, and have clear and complete titles, exact 
though brief column headings, and a short cue 
as to how to read the table. More than this is 
needed. ‘The investigator presumably has done 
much thinking on the subject of the inquiry, 
and should be in the best position to discuss 
the significance of his data. He should briefly 
tell what he thinks each table shows, partic- 
ularly if it seems to indicate a significant trend 
or practice. Any shortcomings of his figures 
should be frankly stated, and possible miscon- 
ceptions by those who may make casual 
spections should be guarded against. Such in- 
terpretation of each table is desirable even 
though very briefly done. 

Tables should be properly totaled and 
checked. The various columns of each table 
should be accurately added and balanced and 
appropriate medians or averages entered. When 
frequency distributions * are involved, the total 
of the vertical and horizontal sub-totals should 
balance. These points may seem obvious, but an 
inspection of a random sampling of tables will 
show that some fail to observe them. 

The totals of columns, the medians, averages, 
or other measures of central tendency are the 
raw material out of which the investigator 
composes the text of his discussion and inter- 
pretation of tables. 

The master tabulation should be included. 
It is usually desirable to present the data from 
a questionnaire in a series of short tables which 
bring out the tendencies and trends on one 
point at a time, but which obscure deviations 
from general practice, and make it impossible 
to identify the data of any specific table in re- 
lation to particular cities or respondents. This 
practice is satisfactory, but something in addi- 
tion is usually desirable, namely, the detailed 
presentation in a master tabulation of the re- 
plies to each item of the questionnaire by indi- 
vidual respondents. Such a tabulation should 
list in the first column the individual respond- 
ents—cities, schools, or individuals—in some 
logical order, which will make for ease in 
locating the data from a given place or person. 
The succeeding columns should be allotted to 








the various questions of the inquiry, the colum, 
heading repeating as fully as practical 
wording of the question concerned. The r, 
of each respondent to each question can then 
indicated. Abbreviations of responses, “‘y”’ 
yes, ““‘n” for no, and other space saving de\ 
meal be used. If the questionnaire is “el 
will be necessary to divide the master tabu 
tion into several sections. 

The ease with which a master tabulation 
takes shape is a good test of one’s success in 
preparing the questionnaire. If the questions 
have been properly prepared,® it will be pos- 
sible to indicate answers in brief form.* If thy 
questionnaire was logically organized in the 
beginning—questions on the same topics being 
grouped together, and sub-questions following 
principal questions—the arrangement of the 
master tabulation will automatically follow 
the questionnaire arrangement. This wil! make 
for ease in tabulating and checking, and wil! 
afford logical dividing points if the question 
naire is so long that the master tabulation must 
be split into more than one section. 

The master tabulation will be of interest + 
but a small percent of those who consult th: 
study and may, therefore, appropriately be sub 
merged at the end of the report. Its inclusion, 
however, may be justified on several bases: ( | 
It has to be made anyway, if the questionnaire 
is to be properly tabulated, so its inclusion in 
volves only the cost of reproduction, (2 
allows other investigators to identify unusual 
or pioneering practices and to investigate them 
more fully, (3) It makes for care in tabulat- 
ing, since the basis for the various tables in 
the main body of the report is found in the 
master tabulation, and if they do not check with 
it, errors may be discovered and corrected. Al! 
respondents are accounted for on each ques- 
tion—whether they reply or not. The reason 
why some respondents did not reply to certain 
questions can be hypothecated. (4) It makes 
for care in reporting. A respondent will hesi 
tate to make careless or inaccurate reports on 
the items of a questionnaire if it is likely his 


1 For an illustration of tables which are relatively complete in themselves, see ‘Practices Affecting Teacher Personn 


Research Bulletin of the National Education Association, Vol. V1, 


No. 4, September, 1928, 


2 See page 81ff of reference No. 24 in the Bibliography for a description of frequency distributions. 


*See the section on the preparation of questionnaires, parti 


cularly pages 15 and 21. 


4 Essential qualifications of answers may be handled in brief footnotes at the foot of the master tabulation. 
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in will return to plague him later in the 
master tabulation. 

Below is illustrated the master tabulation of 
the questions given on pages 15-6 of this bulletin. 

Summaries of the data illustrated in the ac- 
companying master tabulation may be found 
in Tables 4, 5, 11, 12, 13, 14, 15, and 16, of 
“Practices Affecting Teacher Personnel.” Re- 
search Bulletin of the National Education As- 
sociation, Vol. VI, No. 4, September, 1928. 

If for any reason it appears impossible to 
include the master tabulation as a part of the 
report, one should at least give a list of the re- 
spondents to the questionnaire and indicate 
that the complete tabulation is available for 
inspection by interested investigators. 

The questionnaire should be reproduced— 
There should be appended to each report, just 
before or just after the master table, a copy 
of the question blank used in assembling data. 
In evaluating questionnaire data it is fre- 





quently as important to know just how the 


questions were asked as to know the responses. 
When an investigator includes both his blank 
and the master tabulation, all cards are on the 
table and the readers of his report are in a 
position to accept or reevaluate the conclusions 
based on his data. 

Summary—This section offers certain prac- 
tical suggestions regarding the preparation of 
reports based on questionnaire data. Such re- 
ports should be titled and otherwise properly 
identified, the presentation and interpretation 
of data should be carefully organized, a master 
tabulation giving answers of individual re- 
spondents to each question should be included, 
and a copy of the completed questionnaire ap- 
pended. These characteristics of a good report 
although obvious are sometimes lacking. Their 
intelligent observance by investigators would 
improve many questionnaire studies. 


FIGURE III—ILLUSTRATION OF A MASTER TABULATION 
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The Questionnaire Problem From the Viewpoint of the Superin- 
tendent of Schools '* 


Individual superintendents of schools have 
written, and the Department of Superintend- 
ence of the National Education Association 
has resolved, in protest against the number of 
questionnaires which annually reach the offices 
of public school systems. This situation im- 
pelled the Research Division of the National 
Education Association to begin in 1927 an in- 

estigation of the questionnaire problem par- 
ticularly as it concerns the offices of superin- 
tendents of schools. 


Evidence Is Assembled 


In order to secure definite evidence as to 
the size and character of the problem the co- 
operation of a number of state, city, and county 
superintendents of schools was enlisted. The 
school systems asked to cooperate in the study 
were selected to represent various geographic 
areas and school systems of different types. 

These offices agreed, immediately upon the 
receipt of a questionnaire, to mail the Research 
Division a postal-card form? giving the name 
and address of the sender of the inquiry, and 
its subject and date of issuance. Upon receipt 
of one of these notices, a personalized form 
letter was sent to the investigator, asking for 
a copy of his questionnaire and explaining the 
purpose of the request. A follow-up letter was 
sent if a copy of the questionnaire was not re- 
ceived in six weeks. This procedure was con- 
tinued without essential variation for the 
school years 1927-28 and 1928-29. 

At the close of 1927-28, an inquiry was dis- 
patched to each investigator who had furnished 
a copy of his question form, asking informa- 
tion as to the number sent out, the number of 
replies received, and inquiring concerning the 








tabulation and dissemination of results.’ Ne 

a year later, in March, 1929, another inqui 
was sent to all who had supplied copies of the), 
questionnaires seeking information as to 
made of the data they had assembled. 


The Evidence and What It Shows 


The number of questionnaires—Table | 
lists the cooperating school systems and +}, 
number of questionnaires reported in each case. 
After eliminating duplicate cards for the same 
questionnaire reported by different cities a total 
of 376 different questionnaires were reported 
by 14 superintendents of schools in 1927-28. 
In 1928-29, 364 different questionnaires were 
reported by a similar number of cooperators. 

The average annual number of question- 
naires reported by the superintendents’ offices 
of the city school systems in Table 1 is 45. The 
number received by individual cities varies 
widely. Five report less than twenty in a year 
and four report more than seventy-five, tl 
largest report from one city being 137. 

Approximately 370 different questionnaires 
were reported by the 14 cooperating offices in 
each of the two years. This is less than mig!) 
have been expected, considering statements fre 
quently made as to the “floods” of question 
naires which annually descend upon the schools. 
A report from 14 centers, even though ran- 
domly distributed geographically, cannot be 
accepted as a complete showing for the whole 
country, particularly when nearly all of the 
cooperating offices are located in large or mid- 
dle sized cities. It is safe to say, however, that 
it obtained a substantial portion of the ques- 


tionnaires of wide distribution, as well as some 


of those of restricted and local circulation. 


1 Special acknowledgment is due to the following school officials who cooperated in this study by reporting questionnaires |: 


circulation: A. F. Harman, then County Supt. of Schools, Montgomery, Ala.; Fred M. Hunter, then Supt. 
. Research and Statistics, State 
Supt. of Schools, San Francisco, Calif.; C. E. Greene, Director of Research, 
of Education, Hartford, Conn.; Frank W. Ballou, Supt. of 
Kansas; Clarence G. Cooper, County Supt. of Schools, Towson, 
Hills, Mass.; Millard C. Lefler, Supt. of Schools, Lincoln, Neb.; A. L. Hartman, Director of Referenc: 
W, J. Bickett, Supt. of Schools, Trenton, N. J.; William H. Holmes, Supt. of Schools, Mt. Verno: 
nd, Ohio; Julius E. Warren, Supt. of Schools, Lakewood, Ohio; D 

of Schools, Philadelphia, Pa.; J. H. Saunders, Supt. of Schools, Newport News, 
Wis. 


Calif.; W. E. Morgan, Chief, Division of 


search and Surveys, State Board 
Gowans, Supt. of Schools, Hutchinson, 
Supt. of Schools, Wel 
and Research, Montclair, N. J.; 
N. Y.; R. G. Jones, Supt. of Schools, Clevela 
Schools, bs wean Ohio; E. C. Broome, Supt. 

Va.; and Thomas W. Gosling, then Supt. of Schools, Madison, 


Schools, Oakland 
. of Education, Sacramento, Calif.; J. M. Gwin: 


Schools, 
Md.; S. Monroe Graves 


E, M. Otis, Supt 


Others not elsewhere acknowledged who have offered helpful suggestions in the preparation of this Bulletin are: Carter Go: 


Leonard V. Koos, Belmont Farley, and S. D. Shankland. 
2 The form used is reproduced on page 49. 
3 The form used in collecting this i 
4 Form is reproduced on page 51. 


ormation is reproduced on page 50. 
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nver, Colo.; Roger M. Thompson, Director of Re- 
Washington, D. C.; J. W. 
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The number of questionnaires reported by 
‘ndividual cities is of interest. One of the co- 
operating cities reported more than 100 in a 
vear. Lhe average for all cities was 45, or 
about one a week. 

The question should be raised as to whether 
all questionnaires received by the cooperating 
superintendents’ offices were reported to the 
Research Division. It is known that a com- 
plete report was not received from some. On 
the other hand, it is believed that in some 
cities, in which most of the mail passes through 
the hands of one alert secretary, conditions were 
favorable for receiving an approximately com- 
plete report. The five largest reports for a year 
average 89 questionnaires per city. It seems 
reasonable to conclude from the data presented 
that few offices of superintendents of schools in 
the United States receive on the average more 
than two questionnaires a week. 

All of the subsequent conclusions of this 
chapter are dependent upon an analysis of a 
sampling of the questionnaires which reach 


TABLE 1—NUMBER OF QUESTION- 
NAIRES REPORTED BY COOPERAT- 
ING SCHOOL SYSTEMS 


























Number of 
questionnaires 
Cooperators saint in 
1927-28 | 1928- 29 
1 2 3 
State Departments 
es 5 5 e.epit HAG bd06-6 30 als 
en hose ceve hse 41 17 
City School Systems | 
Oakland, California*........... 76 a 4 
San Francisco, California**..... ay 87 
Soares, he eed 29 
Washington, D. C.*........ : 14 P 
Hutchinson, Kansas.......... 35 21 | 
Wellesley Hills, Massachusetts**. bw 9 
Lincoln, Nebraska** ; aa 38 
Montclair, New Jersey. . ci 41 27 
Trenton, New 17 48 
Mount Mig New York...... 28 16 
| Cleveland, Ohio............... 45 63 | 
Lakewood, ee hess 36 
| Willoughby, 0 ee 4 ee 
| Philadelphia, Pennsylvania**... . ~~ 81 
Newport News, Virginia........ 38 36 
Madison, —— Ghiattne wees 137 con 
Come School S | 
ra eng cate, Alabama’*. 3 eT 
altimore County, Maryland*.. 1 ae | 
wo. ad, Fs EE eae ina 15 
Total number reported. ..... 510 $23 
Total number, excluding du- 
EL Gin at oinc.ed 66.606 376 364 








*Reported in 1927-28 only. 
**Reported in 1928-29 only. 




















the offices of superintendents of schools. It is 
believed that this sampling 
dom to justify the conclusions which are drawn. 


is sufficiently ran- 


In the cases where conclusions may have been 
significantly affected by non-random selection 
special notice of this possibility has been taken. 
A total 
of 376 different questionnaires were reported 
by the 14 centers cooperating in 1927-28. It 
was possible to secure copies of 306 of these. 
The 70 remaining questionnaire circulators 
failed to respond to two separate requests ask- 
ing for copies. Of the 306 received, 39 were 
eliminated for one of these reasons: (1) 
complete copy was sent, (2) the inquiry was 
merely a general request for such material as 
was available on a given topic, and not a true 
questionnaire and (3) the request was of the 
nature of a regular periodic report required by 
the state department of education. Similar 
treatment of the 364 questionnaires reported 
for 1928-29 reduced the number to 265. There 
were thus available 267 and 265 bona-fide ques- 
tionnaires for 1927-28 and 1928-29 respec- 
tively, or a total of 532, certain features of 
which could be studied in detail. 

Although 740 inquiries were reported by the 
cooperators in the two years, only 532 could 
be studied in detail. TCo what extent did this 
reduction involve the factor of unfair selec- 
tion? The reduction was due to two causes, 
first, the elimination of certain inquiry forms 
which were not bona-fide questionnaires. Non- 
random selection did not enter here. Second, a 
small percent of the originators did not respond 
to requests for a copy of their blank. It is likely 
that the failure to respond was due in some 
cases to the fact that the blanks had been 
poorly prepared. Some selection probably en- 
tered because of this and other factors. Copies 
of approximately eighty percent of the inquiries 
reported were obtained. Non-random selection, 
therefore, could not have been present except 
to a minor degree. 

Who sends questionnaires?—Table 2 sum- 
marizes the data as to where questionnaires 
originate. Of 532 questionnaires which reached 
the offices of certain superintendents of schools 
over a two-year period, 27 percent were sent 
out by a city superintendent of schools or by 
members of his central office staff, i. e., assist- 
ant superintendent, director, or supervisor. An 


Analysis of questionnaires received- 


an in- 
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additional 23 percent were sent out by other for considerably more than half of those s: 
employees of school systems, i. e., principals, ied. That this is not merely a reflection of 


officers of teachers’ organizations, or individ- fact that administrative problems account F 
uals. Therefore, half the questionnaires were a large part of all educational research, js jy. | 
sent out by superintendents or members of their dicated by Koos’ data in Table 3, which shows 
executive or instructional staffs. that 54 percent of printed questionnaire stwi\ies | 
Approximately 23 percent were sent from deal with administration, while but 31 percen: | 
the colleges by professors and graduate students. of published investigations which do not em : 
Private education associations and foundations ploy the questionnaire method are in the field ’ 
a 





originated 10 percent. The remainder, about of school administration. 

17 percent, originated from a variety of agen- It seems safe to conclude therefore, that the 

cies. questionnaire is of especial importance to those © 
Half the job of regulating the output of interested in educational administration, 

questionnaires which reach the offices of super- whether they be research workers or schoo! 

intendents of schools, therefore, must be per- executives. The abolition of the questionnaire 





formed by the superintendent and members of would close one of the important doors through T 
his executive or instructional staffs. Superin- which students of school administration, }ot) 
tendents and their staffs, college professors, theoretical and practical, secure data essential  r 
and representatives of a few educational asso- to their purposes. 
ciations and foundations by entering into some When are questionnaires sent out?—Oc. 
informal, cooperative plan, such as that sug- tober to April seems to be the “open season.” | 


gested on page 38, can control the situation as None of the months during this period of the 
it affects more than 80 percent of the inquiries year seems to have a corner on the supply, | _ 
which reach the offices of school executives. according to the data of Table 4. In organiz- 
The field of the questionnaire—In order to ing provisions for handling questionnaires in a 
discover the educational fields from which these superintendent’s office, it would seem wise to 
questionnaires sought information, they were _ set up facilities that will be available through 
classified under a number of major educational out the school year. 
heads. Table 3 summarizes the result, and How long are questionnaires?—The num 





compares it with the findings of Koos who _ ber of separate questions asked, or items of in 
analysed 143 published questionnaire investi- formation requested, was taken as an indication 
gations as to fields dealt with. Both investiga- of the length of questionnaires. According to | 
tions agree that administrative topics account this measure, questionnaires vary in length 





TABLE 2-WHO SENDS QUESTIONNAIRES TO THE OFFICES OF SUPERINTEND- 
ENTS OF SCHOOLS? 




































































- —— —=— — ¥ 
1927-28 and } 
1927-28 1928-29 1928-29 i a 
Originator of questionnaire 
Number Number Number 
cent Percent cont Percent sent Percent 
| 1 2 3 4 5 6 7 
City superintendent or member of central office i 
ff 66 24.7 78 29.4 144 27.1 : 
62 23.2 60 22.7 122 22.9 
61 22.9 61 23.0 122 22.9 
25 9.4 30 11.3 55 10.3 
17 6.4 8 3.0 25 4.7 
10 3.7 9 3.4 19 3.6 
7 2.6 10 3.8 17 3.2 i 
19 7.1 9 3.4 28 3.3 
Be amnaans 6terdevinetecdetesh taewees 267 100.0 265 100.0 532 100.0 
| 
Read table thus: Of 267 different questionnaires received by offices of superintendents of schools in 1927-28, 66, or 
24.7 percent were sent out by city superintendents of schools or members of central office staff; 62, or 23.2 percent were 
sent out by principals, teachers, or other school employees, etc. Similarly read data for 1928-29 and for both years combined. q 
i 
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from those which list less than five items to 
those which include more than 500. Some are 
half sheets, others are almost volumes. ‘The 
median number of items requested by the 267 
questionnaires of 1927-28 was 17.4. Over sixty 
percent had 20 items or less. Table 5 gives fur- 
ther data on this point. Most questionnaires 
seem to be reasonable in their demands insofar 
as number of items requested is concerned. 
An inspection of the individual question- 
naires did not suggest that brevity or length 
should be adopted as an infallible criterion in 
deciding whether a questionnaire is worthy of 


answer. Some very short and other very long 
ones were well prepared and deserved careful 
answering. On the other hand there were 
atrocious examples of both long and short ques 
tionnaires of the type which deserved to be 
assigned to the waste basket or returned un- 
answered 
letter. 


accompanied by an _ appropriate 

Form of the questions—It has been urged 
that time could be conserved through the use 
of questions which may be answered yes or no, 
or by the use of check marks. The 267 ques- 
tionnaires were examined to determine the form 


TABLE 3.—-THE FIELDS OF EDUCATION IN WHICH QUESTIONNAIRES SEEK 
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INFORMATION 
Questionnaires to Published question- Pare a 
superintendents in | naire investigations | “ ti aan a odes ww 
= 1 
Field 1927-28 and 1928-29! | analysed by Koos? by Koos ! 
Number Percent Number Percent Number Percent 
1 2 3 4 5 6 7 
a? Administration ee iG ore msde seats 398 74.8 77 53.9 136 1.1 
| 2, Curriculum. ' 91 17.1 35 24.5 49 11.2 
3. Methods of teaching. . a | 7 4.9 56 2.8 
4, Pa pw ec rceescccces 10 1.9 | 4 2.8 1 2 
5. Tests and measurements. . RS 12 | 8.4 98 2.3 
6. Paychology............ ie 7 he — & | 5.6 47 10.7 
7. Statiotics...... ieatueks 5 9 uae san 33 5 
8. Philosophy..... ~ ‘ nat j 4 9 
9. Research...... SSP ee 8 oe | 2 | 5 
| 10, Combinations. . ; 12 2.3 | .. 12 | 27 
SR ae ee _ | 532 100.0 | 143 100.0 438 99.9 











larly read data for columns 4 and 5, and 6 and 7. 





Read table thus: Of 532 questionnaires sent to superintendents in 1927-28 and 1928-29, 398, or 74.8 percent sought 
information concerning administration; 91, or 17.1 percent sought information concerning the curriculum, etc. Simi- 


! The Research Division has available detailed tables on which columns 2 and 3 are based. 
2 Koos, Leonard V. The Questionnaire in Education, Macmillan Company, New York, 1928, page 32. 








TABLE 4.—MONTH OF ISSUANCE OF QUESTIONNAIRES SENT TO OFFICES OF 


SUPERINTENDENTS OF SCHOOLS 














Issued in 1927-28 | Issued in 1928-29 


Month issued ote eae 
Number 

1 2 
CO Sere rr pr ee yt x 
October..... rp pene 18 
November a hak ; 41 
December 38 
January. 43 
February 35 
March..... 39 
April...... 27 
fay. . 12 
Ss ce tisaas 6 
Total. . 267 


Total 


_ Number | Percent 


100.0 | 265 100.0 


Percent | cent | Number Percent 
3 hi 2 ik 6 7 
—EEE —— —_— —— = —_ - - = = —_ — _ 
| 3.0 “st ort = 6.4 
6.7 41 ; ea 59 | wa 
; 2a | 39 14.7 | 80 | 15.0 
: ee * i 2 | 8.3 60 } 11.3 
| pe 29 | 10.9 72 13.5 
he <a a eee es 54 10.2 
| ae.) 2 4.0 76 14.3 
the 31 ‘7 58 10.9 
} 4.5 19 7.2 31 5.8 
2a 2 8 8 1.5 
| | $32 100.0 














Read table thus: Of 267 different questionnaires sent to offices of superintendents of schools in 1927-28, 8, or 3 percent 
were issued in September; 18, or 6.7 percent were issued in October, etc. Similarly read data for 1928-29 and for total. 
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TABLE 5.—DISTRIBUTION OF _ 267 
QUESTIONNAIRES CIRCULATED IN 
1927-28 ACCORDING TO NUMBER OF 


























ITEMS OF INFORMATION RE- 
QUESTED 
Number of items penn hy Percent | 
| requested naires of total 
a 1 a 2 | 3 
| 2s 38 14.2 
6-10 48 18.0 
} 1-18.. 37 13.9 
16-20 38 14.2 
21-25 16 6.0 
26-30 il 4.1 
31-40 25 9.4 | 
41-50 14 $.3 
51-60 6 2.2 
61-80. . 8 3.0 
81-100 5 1.9 
101-200. 9 ©? 
201-S00..... 8 3.0 
More than 500 4 Sue 
| Ce a 100.0 | 
Median number of items... . 17.4 
Read table thus: Of 267 different questionnaires 
sent to superintendents in 1927-28, 38, or 14.2 percent | 
requested information on from 2 to 5 items; 48, or 18 
percent requested information on from 6 to 10 items, | 
etc. 
} 
— ee } 














of question used. Four types of questions were 
recognized, according to the form of the re- 
sponse desired, as follows: (1) Yes-or-no ques- 
tions, those calling for a direct categorical an- 
swer, (2) check questions, giving respondents 
the opportunity to check or underline one of 
several answers or symbols of rank value given 
on the questionnaires, (3) short-answer ques- 
tions, requests for statistical information or 
short verbal responses, and requests that the 
respondent enter a number or a letter indicat- 
ing the rank or weight of various propositions ; 
and (4) essay questions, those calling for 
lengthy verbal responses of descriptive or ana- 
lytical nature, which could not be subjected to 
statistical treatment. 

It was found that 70.5 percent of the ques- 
tionnaires consist of combinations of the above 
types of questions, and that 29.5 percent use 
some one type of question exclusively. The vari- 
ous types appear with the following frequency: 
yes-or-no, and short-answer, 34.5 percent; 
short-answer only, 23.2 percent; yes-or-no, 
check, and short-answer, 14.2 percent; yes-or- 
no, short-answer and essay, 7.5 percent ; check 
questions only, 4.1 percent; check and short- 
answer, 4.1 percent; and other types and com- 
binations, 12.4 percent. Essay questions are in- 





cluded in 15.8 percent of the forms, Only 8. 
percent of the questionnaires are limited 
yes-or-no and check questions, but 64.9 p, 
cent use them together with other types , 
questions. 

The foregoing data indicate that approx 
mately 85 percent of the inquiries sent to sy 
perintendents consist of questions which can 
be answered with a minimum of writing, as 
only 15.8 percent include questions which cal! 
for extended answers of the essay type. Al! of 
the short-answer questions were not as clea; 
cut as they might have been, and some could 
be criticised from other points of view. [t is 
encouraging to note, however, that compara- 
tively few questionnaire senders are asking 
questions which call for extended answers of 
the type which first try the soul of the respond- 
ent and later defy efforts at tabulation. 

Character of information sought—\t has 
been stated that these types of information 
may legitimately be collected by questionnaires: 
(1) Facts known to the person responding, 
(2) facts readily available from school records, 
and (3) judgment or opinion. Of the 267 
questionnaires of 1927-28, 51.5 percent asked 
for facts or figures only, 10.6 percent requested 
judgments and opinions, while 37.9 contained 
items asking for both. 

The chapter of this Bulletin dealing with 
the characteristics of good questionnaires pre- 
sents certain material bearing on the character 
of questions asked. Here it is sufficient to say 
that questions of fact should be limited to items 
which are ascertainable with a reasonable ex- 
penditure of time and effort. When questions 
do not meet this criterion they should be left 
blank. On the other hand, there are questions 
of fact which some school systems cannot an- 
swer which they should be able to answer. For 
example some systems have no record of their 
employed personnel on such essential items as 
training, experience and age. Such information 
automatically accumulates if new employees 
are required to fill out a personnel card." 

The difficulty with some questionnaire items 
therefore, is not that they are improper items, 
but that some school systems do not maintain 
the records which principles of sound manage- 
ment would justify. It is suggested, therefore, 
that when a carefully prepared questionnaire 
from a competent investigator requests items ot 


1 See page 274 of “‘School Records and Reports,” Research Bulletin of the National Education Association, Vol. V, No. 5 
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factual information concerning a school sys- 
rem. which cannot be obtained except by a spe- 
cial investigation, it may be desirable to eval- 
te one’s system of records.’ 

Of the 48.5 percent of questionnaires in- 
cluding some requests for judgment and opin- 


u 


ion. a large proportion asked that certain sub- 
‘ective factors be ranked or weighted. This 
type of questionnaire, whereas effective if care- 
fully prepared, is often especially resented by 
respondents. To answer properly such a ques- 
tionnaire frequently not merely takes time but 
it requires the time of those in the more re- 
sponsible positions. It cannot be answered by 
clerical workers. This, as well as other opinion- 
seeking questionnaires, should be used as little 
as possible and then only by experts. This 
whole problem is discussed at greater length 
in the section dealing with the characteristics 
of good questionnaires. 

Duplication in questionnaires—The amount 
of duplication in questionnaire circulation gen- 
erally has been assumed to be large. It has been 
asserted that tremendous waste results from 
the fact that the same questions are asked by a 
number of different questionnaires. To ascer- 
tain the amount of duplication, the 267 ques- 
tionnaires collected in 1927-28 were inten- 
sively studied with particular reference to the 
number of times identical questions occurred 
in different questionnaires. Questions were 
classed as identical if they called for identical 
answers even though somewhat differently 
worded. The amount of duplication was found 
to be small. To be sure, 40 of the 267 ques- 
tionnaires dealt with phases of the salary prob- 
lem. But the number of times the same salary 
question was asked by more than one question- 
naire was not great. The largest duplication 
was the request by seven different question- 
naires for the minimum and maximum salaries 
of both elementary and high school teachers. 
Those dealing with the curriculum similarly 
revealed few identical questions. 

Comparatively little duplication of ques- 
tions was found on any of the topics. Even 
when on the same topic, the approach varied 
and the particular information requested 
differed. 

The results of the tabulation of duplicating 
questions is shown in Table 6. It shows that 26 








questions were found which occurred on three 


or more questionnaires. Such was the duplica- 
tion discovered on 267 blanks, with a total of 
13,070 questions. 

This small amount of actual duplication 
does not mean that there is no waste in ques- 
tionnaires. If forty salary questionnaires give 
a poorer statistical picture of the salary situa- 
tion in the United States than the one com- 
prehensive study which the National Educa- 
tion Association makes biennially, then it is 
double waste to send out the forty, from forty 
different centers, with their limited means of 
composing, circulating, following up, tabulat- 
ing, and interpreting these blanks. From this 
fact it is not concluded that the circulation of 
questionnaires should be centralized in a few 
hands. The problem is one which calls for dis- 
crimination as to function. To say that there 
are certain types of inquiries which can be most 
carried on by 
agencies and which ordinarily should be at- 
tempted only by such agencies, does not imply 
that there are no investigations which indi- 
viduals, and local centers can pursue with 
profit. Opportunity to circulate questionnaires 
should not be closed to the individual, but the 
individual investigator should use discrimina- 
tion in selecting subjects to be studied by this 
method. 

Factors affecting percent of questionnaires 
answered—Information was sought from the 
267 questionnaire circulators of 1927-28 as to 
the number of questionnaires sent out and re- 
TABLE 6.—NUMBER OF DUPLICATING 

QUESTIONS IN 267 QUESTIONNAIRES 


ASKING A TOTAL OF 13,070* QUES- 
TIONS 


economically well-equipped 








Number of questions Frequency of } 
appearing more than twice occurrence 
1 2 

2 7 

2 6 

10 5 

5 4 

| 7 3 

ae. seen teeeainticatatieena tia entities < ms 
cS re 2 








} * Calculated by multiplying mid-points of intervals 
| (Column 1, Table 5) by number of cases (Column 
2, Table 5) and adding. 

| Read table thus: There were two questions each of 
which reoccurred in seven questionnaires; two ques- 
| tions each of which reoccurred in six questionnaires, 
| ete. 

} 











'The Research Bulletin cited in the previous footnote is a helpful source of information on records and reports. 
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It was possible to secure a re- 


plies received. 
port from 136. The fact that only a little over 


responded 
may be due to several factors, one of which is 
probably that some of those who had a poor 
reply did not wish so to report. The conclu- 
sions based on the 136 replies need to be in- 
terpreted with this in mind. The succeeding 
facts about the percent of reply received 
probably for a selected group of the better 
questionnaires. 

Table 7, indicates that the typical question- 
naire of the 136 received a 69 percent reply. 
The reply varied somewhat according to the 
position held by the investigator. As might be 
expected, superintendents of schools are some- 
what more likely to answer inquiries origi- 
nated by other superintendents than by other 
persons. However, an examination of the de- 
tailed distributions of replies, not given here,? 
shows that the percent of reply received de- 
pends more upon other factors than upon the 
originator. Some questionnaires sent by super- 
intendents received a reply of less than a forty 
percent, and others a 100 percent reply. The 
same may be said for those of college professors 
and other classes of originators. 

The month in which a questionnaire is is- 
sued also has some influence upon the reply 
received, according to Table 8. Generally 
speaking, it appears that the earlier in the 
school year a questionnaire is issued the more 
TABLE 7.—PERCENT OF REPLY RE- 


CEIVED BY VARIOUS CLASSES OF 
QUESTIONNAIRE CIRCULATORS 


half of the questionnaire-makers 
































Number Median 
Originator of questionnaire of ques- percent 
tionnaires | of reply 
1 2 3 
City eee or member 
central office. ............. 42 75.4 | 
School oelasionl ¢ or other school | 
—— pean yer 23 66.0 | 
College professors or students. .. 29 60.4 
Private association or foundation. 19 58.0 
SE EE ae 8 61.0 
IDL cage Sw6 6-0-5 duns Cnees 6 71.0 
I. a ve-en clap eraswned 9 78.5 
yp eee ee 136 69.0 
Read table thus: The median percent of reply to 
the 42 different questionnaires pe out by cit en ornee 
intendents of schools or members of central o' 
75.4 percent; median percent of reply to the 23 different 
questionnaires sent by school principals or other 
school employees was 66 percent. etc. 





ee 











TABLE 8—PERCENT OF REPLY Rp. 
CEIVED AS RELATED TO MONTH 
OF ISSUANCE OF QUESTIONNAIRES 
































Number Median 

Month of issuance teaued aennent 

of reply 

1 s a @ 

September...... ; : uate 5 76.0 
October...... ; ~ae ait 12 74.3 
November. ....... , 22 73.5 
mber..... cehas ‘ 19 76.0 
January...... aah wan 20 59.0 
ae Sad - 15 69.8 
SE rise 16 66.0 
_ See ae ; aire 17 64.8 
0 ae ae ~~ 10 61.0 
| Bs iiedd cnkesedeaees 136 69 .( 
Read table thus: The median percent of reply t 

5 different questionnaires issued in September was 
76 percent; the median percent of reply to the 1? 

different questionnaires issued in October was 74.3 

percent, etc. 








likely it is to receive attention by a superin- 
tendent’s office. January seems to be the poores: 
time to cast a questionnaire upon the waters. 
Table 9 presents data as to the relation of 
the length of the questionnaire and the reply 
received. Generally speaking, the shorter the 
questionnaire, the better the reply. This rela- 
tionship is far from invariable, however, for 
the questionnaires asking for five items or less 
received a reply all the way from less than 20 
percent to 100 percent, while the longest of 
the 136, a veritable volume asking for over 
1800 items, received a reply of 88 percent. 
TABLE 9.—PERCENT OF REPLY RE- 


CEIVED AS RELATED TO LENGTH 
OF QUESTIONNAIRE 





























Number of items of Number | Median 
information requested of ques- | Percent 
tionnaires | of reply 
1 2 3 
OO ree re 20 78.5 
See 16 71.0 
A ee 22 71.0 
ea 17 68.5 
Des ha.e x 'oe one ie 15 66.0 
Sa 14 61.0 
| ee 6 71.0 
EL, Sen oye iee we bs 10 76.0 
a es eee 5 66.0 
os bike tas wh em ew ee g& 54.3 
§01-1000............... 2 , 
1800-1900 1 
» ne Aa A See 136 69.0 
Read table thus: The median percent of reply to 
20 different questionnaires requesting information « 
5 or less items was el ay mw the median percent of 
reply to the 16 req information on from 6 to 
10 items was 71 percent, etc. 














1 These tabulations are on file in the Research Division. 
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The reply received is also affected slightly 
by the character of information requested. ‘The 
71 questionnaires asking for objective data 
only, received a median reply of 74 percent, 
the 51 asking for both objective and subjective 
data received a 65 percent reply, and the 14 
asking wholly for subjective material received 
a 61 percent reply. 

Availability of questionnaire results—In 
March, 1929, information was sought as to the 
availability and use made of the tabulations 
and conclusions based on the 267 question- 
naires of 1927-28. Information on these points 
was obtained for 203 of the 267. The studies 
from these 203 questionnaires were in the fol- 
lowing conditions: 


16 or 7.9 percent—not tabulated or used 

16 or 7.9 percent—tabulation still in progress 
18 or 8.9 percent—in long hand only 

56 or 27.6 percent—typewritten 

47 or 23.1 percent—mimeographed 

50 or 24.6 percent—printed or to be printed 


203 100.0 percent 


The foregoing tabulation shows that a year 
after most of the 203 questionnaires had been 
sent out, only 97 or less than half of them had 
been put in mimeographed or printed form so 
that the results could be generally circulated. 
It also appears that at least 16, and possibly 32 
of the 203 were not to be tabulated. It also is 
likely that little progress in tabulating had been 
made by many of the 64 of the 267 who would 
not report on this matter. 

This evidence seems to indicate that at least 
half of the questionnaires which reach the of- 
fices of superintendents are not put in such form 
that they may be circulated among respondents 
or other interested people. 

Uses made of questionnaire studies—The 
267 questionaires of 1927-28 were studied as 
to what use was made of their results. No re- 
port could be obtained on this point for 64 of 
the 267. The presumption is that many of these 
64 were not put to any significant use. Facts 
were obtained on the remaining 203 of the 267. 
Sixteen of the investigators stated that the 
replies to their questionnaire had not been 
tabulated and that no use had been made of 
them and 16 others reported that nothing had 
been completed by March, 1929. One hundred 


and seventy-one studies had been completed and 
used in various ways, as shown in Table 10). |» 
several cases a questionnaire was used for m 
than one purpose, consequently the reported 
number of uses exceeds the total numbe: 
questionnaires. Of the 171 completed studie. 
34.5 percent were used in connection with loc: 
school problems ; 8.8 percent were used in work 
ing on a problem of a state school system; | 4, 
percent were used in reports of state or na 
tional organizations; 15.2 percent contributed 
toward the preparation of graduate theses; | || 
percent for other papers and reports connect 
with college work; 12.9 percent were used jn 
speeches; 19.3 percent were used in magazin 
articles; 1.8 percent were used in books; 3.5 
percent were used in advertising; and 9.4 pe: 
cent were used for other purposes. 

It is evident that no use is made of the results 
of some of the questionnaires which reach the 
superintendent's office. Effort expended in an 
swering such inquiries is wasted. ‘The use mad 
of certain other inquiries is narrow—benetiting 
none but the sender of the inquiry. On the 
other hand a substantial portion of the blanks 
contribute to a variety of valuable outcomes 
The use mentioned most frequently is that ot 
preparing reports on local school problems. 
Forty-three percent of the 203 questionnaires 
contributed to the solution of state and |i 
school problems. 

It has been stated that the majority of ques 
tionnaires collect data for college theses or term 
papers. The facts do not support this belief. |: 
was found that 15 percent of the 171 ques 
tionnaires were used in theses and 11 percent 
in other college work. Some of this 26 percent 
were also used for other purposes. 

Some of the inquiries were issued by scho 
officials who deplored the necessity of using @ 
questionnaire, but who felt that the need for 
information was imperative. Further collection 
and distribution of information on school prac- 
tice by state and national agencies would re- 
duce the number of such questionnaires. 


The fact that 22 of the questionnaires were 
used in speeches and 33 in magazine articles 
does not mean that they were used for those 
purposes alone. In the majority of cases, ques 
tionnaires used in lectures or articles were als 
used in connection with local problems or « 


[ 36] 








Rt le a Pe Sat 


ee 


5 ila 





| 
| 
| 








q 
th 


pl 
en 
qu 
Sv 


bl; 


cit 
po 
sel 


re] 





hose 


jues- 


alse 


yr as 














Sere tedk 








acne 


thesis material. The lectures and articles ap- 
pear to be by-products of studies made for 
other reasons. In a few cases, however, the only 
use made of the questionnaire data was in a 
speech or article. 

Of the various uses made of questionnaires, 
some will seem more worthy than others to 
respondents. The investigator who sends with 
his questionnaire a statement of his reasons for 
making the study will give the respondent a 
basis on which to decide whether or not it is 
one which he wishes to support, and will im- 
prove the percent of return from those in sym- 
pathy with the project. 

Summary and conclusions— This section re- 
ports and interprets the results of evidence 
assembled in studying the size and character ot 
the questionnaire problem as it affects the offices 
of superintendents of schools. Over a period of 
two years several hundred different question- 
naires received by the offices of superintendents 
were collected, analysed, and followed up. A 
smaller number of questionnaires was dis- 
covered than was expected. Fourteen offices re- 
ported approximately 370 different question 
naires in each of two years. Only one cooper- 
ating city reported more than 100 in a year. 
The average per city was 45, or about one a 
week. It appears that few offices of superin- 
tendents of schools receive more than two ques- 
tionnaires a week. Half of those which reach 
school offices are sent out by superintendents of 
schools or members of their executive or in- 
structional staffs. Considerably more than half 
deal with administrative topics. 

October to April is the ‘open season” for 
questionnaires. A majority of them are rela- 
tively short, requesting less than 20 items of 
information, most of which can be supplied by 
a brief, objective answer. The amount of du- 
plication was found to be small. In 267 differ- 
ent questionnaires, asking a total of 13,070 
questions, 26 questions calling for identical an- 
swers, occurred on three or more different 
blanks, 

A reply of 69 percent was received by 136 
circulators of questionnaires, reporting on this 
point, who probably constitute a somewhat 
selected group. A somewhat higher percent of 
replies was usually received on those orginat- 
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ing from official sources, issued early in the 
school year, and which were short. 

Less than half of those who send question 
naires to offices of superintendents of schools 
put the results received in such form that they 
can be generally circulated to respondents. 
Such reports as could be obtained as to use 
made of questionnaires, indicate that 43 per 
cent are employed in the solution of state or 
local problems of school administration. Ap 
proximately 25 percent are used in the prepara 
tion of graduate theses or similar college work. 
The results from many questionnaires are used 
for several valuable purposes. No use is made 
of the results obtained by a considerable pet 
cent. 

The data presented seem to indicate that the 
seriousness of the problem of answering ques- 
tionnaires has been somewhat exaggerated. 
This is not to be wondered at. When one is 
weighed down with pressing administrative du 
ties even a few poorly conceived questionnaires 
are apt to disturb one’s equanimity. Even 
though the number of questionaires is some- 
what smaller than has sometimes been stated, 
it is not implied that the burden of dealing 
with these inquiries is of little weight. Rather, 
it is of sufficient consequence to suggest that 
particular provision be made for this purpose. 
Such provision should be carefully planned 
along the lines suggested in a subsequent sec- 
tion. Even a minimum amount of professional 
and clerical service can accomplish much in 
dealing with questionnaires if it is well directed. 
If this is done most of the waste which now 
enters into the questionnaire method of in- 
vestigation, and there undoubtedly is consider- 
able, can be eliminated. 

It is realized that many superintendents’ 
offices are inadequately supplied with clerical 
help. One of the best ways of securing such 
help is to convince school boards and com- 
munities that such help is essential to the busi 
nesslike operation of a school office, and to em- 
ploy it for significant purposes when it is 
granted. The prompt and accurate answering 
of worthy questionnaires should be provided 
for by all properly equipped offices. Some pro- 
vision for discharging this obligation even 
though it may be small, is eminently justified in 


the light of evidence presented elsewhere in 


J 





F 





eee 


this Bulletin. A general failure to provide for 
the efficient discharge of this duty, would cut 
off at the source one of the springs from which 
is drawn the factual information essential to 






efficient school administration. No single grou, 
of school people should be more directly jp 
terested in preventing such a calamity ¢! 
those engaged in school superintendence. 


A Proposed Plan For the Cooperative Regulation of Questionnaires 


It was stated in an earlier section that the 
solution of the questionnaire problem lies not 
in abolition but in proper regulation through 
voluntary, cooperative action on the part of 
those who send and receive questionnaires. The 
purpose of this chapter is to suggest a plan 
through which such action may effectively 
function. 

As was pointed out above, over 80 percent 
of the questionnaires studied in this investiga- 
tion were originated by superintendents of 
schools and their staffs, college professors, or 
representatives of a few educational associa- 
tions or foundations. These three groups have 
it within their power to correct the undesirable 
features of the questionnaire method of investi- 
gation. This may be done by working along 
the following lines. 

Need of a policy and office plan for handling 
incoming questionnaires—An office routine for 
handling questionnaires will save much time. 
Such a plan is needed in the central offices of 
school systems and in other offices which re- 
ceive questionnaires. Just what the details of 
this plan are is less important than that there 
should be one. Many offices otherwise well or- 
ganized show little efficiency in this regard. 

An effective plan for dealing with incoming 
questionnaires will usually require that one 
person be made responsible for deciding as to 
which ones are to be answered, for securing 
the information requested when the decision is 
to answer, for returning them to the sender, 
and for keeping a copy of data supplied. 

In a small school system the superintendent 
working through a clerk or other assistant will 
necessarily exercise these functions. In a larger 
organization, this responsibility could appro- 
priately be delegated to the director of research 
or some other properly qualified person. 

The definite fixing of responsibility for deal- 
ing with questionnaires should not mean it 
should become a one man affair. Rather it 
should mean that some one is made responsible 


for coordinating and conserving such energy 
as is available for this purpose. When this js 
not done waste is likely to result. In one in 
stance, where the route of a questionnaire 
through the offices of a superintendent’s staff 
was traced, it was found that no less than | 
different offices and eight different people had 
spent some time on the questionnaire, and iad 
in each case cleverly “‘passed the buck” on to 
the next person. Although in toto more than 
enough time to answer the questionnaire had 
been consumed, it still was only partly done. 
Even more startling was the unanimous deci 
sion on the part of the various people ~who had 
a try at answering it, when they were called 
together on the matter, that the questionnaire 
was not worth answering. Poor administra- 
tion is a soft term for such a situation. A better 
way is to make some one person responsible for 
shepherding all questionnaires, and to give him 
proper guidance in the form of abriefstatement 
of the policy of the particular organization as 
it concerns answering questionnaires, and tw 
see that responsible members of the stati un 
derstand the policy. 

When this is done the answering of ques- 
tionnaires becomes a recognized function with 
a dignified status whereby the agency concerned 
makes some contribution to the building up ot 
educational knowledge, and in return receives 
dividends of information helpful in shaping 
educational policies and procedures. ‘The pro- 
fessional and clerical service alloted to this 
task may be relatively small, but the allotment 
should be definite and should enjoy a recog 
nized status. To allow this duty to remain 
everybody’s job is likely to result in the con 
sumption of much effort with little benefit to 
anyone. 

The person made responsible for dealing 
with questionnaires will need to set up a plan 
for deciding which ones are worthy of answe: 
He will need to have in mind certain criteria 
in reaching prompt and proper decisions on 
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point. A rough rating plan similar to that 
en on the next page should be useful. The 
estions of this plan may be briefly applied as 


Y 


‘ndicated below, proper modifications being 


nade to meet the particular situation. 
1. Is the questionnaire Spon- 
red?—This question is discussed on page 
14 of this bulletin. In general questionnaires 


adequately 


from responsible agencies such as those cited 
on page 14 should take precedence over those 
sent out by individuals or by agencies of no 
particular standing. There is a greater likeli- 
hood that the questionnaire has been well pre- 
pared if it comes from a well organized office, 
and a better chance that replies will eventuate 
in something useful to the respondent. 

2. Is the purpose of the study frankly stated, 
and is it one which calls for a reply under the 
policy set up for dealing with questionnaires? 


—This subject is discussed on page 23 of this 
Bulletin. One should be generous in applying 
this criterion. 

Questionnaires which assemble data to be 
used in’ writing graduate theses, and partic- 
ularly those which collect data for “term 
papers’ have been generally condemned. The 
objection to the term paper questionnaire is 
based on good grounds. It is an imposition to 
ask busy 


ordinary 


people to take time to reply to the 
“half-baked” questionnaire sent out 
for this purpose. Such questionnaires usually 
promise benefit to no one but the sender and 
frequently not even to him. 

No such general interdict is justifiable in 
regard to graduate theses. If the question- 
naire meets the criterion of the rating plan it 
should be answered. It is doubtless a fact that 
this route to the Ph. D. has been overworked 











FIGURE IV 
An Informal Rating Plan For Questionnaires 
If after inspecting a questionnaire, one may answer yes to each of the following 
questions, it should be answered. If no is the answer to several of the questions the 
questionnaire should be promptly returned to the sender unanswered. For further 
elucidation of these criteria see discussion above. 
1. Is the questionnaire adequately sponsored ?______________ Yes—— No—— 
2. Is the purpose of the study frankly stated, and is it one 
which calls for a reply under the policy set up for dealing 
ETI ET LOT ET Yes— No- 
3. Is the questionnaire on a worthy educational topic ?_____- Yes—— No 
4. Is the questionnaire well organized ?______.____________ Yes—— No—— 
5. Are the questions clearly and briefly worded ?_________- Yese——  No— 
6. Can most of the questions be briefly answered with a check 
mark or by a fact or figure, and is the number of questions 
requiring extensive subjective replies kept to a minimum?_. Yes—— No—— 
7. Is the information requested not available elsewhere, and 
obtainable only through questionnaire ?____.___________ Yes— No. 
8. Is the questionnaire set up in proper mechanical form?_... Yes—— No—— 
9. Are the demands of the questionnaire reasonable?__._..... Yes—— No- 
10. Is a summary of results or other proper return promised 
ES EEE ES SE a Ee ee EE Yes—— No——— 
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and that some questionnaires sent out in con- 
nection with graduate theses have been poorly 
conceived and poorly prepared. On the other 
hand some valuable studies have come from 
this source. Schools of education are recogniz- 
ing the need of regulating this phase of the 
questionnaire problem along the line of the 
suggestions made on page +1. When this is 
done the justification of a general interdict 
against the collection of data for use in ad- 
vanced dissertations largely disappears. 

3. Is the questionnaire on a worthy educa- 
tional topic?—This criterion is discussed at 
length on page 14. If a summary of the results 
of the questionnaire would be valuable to the 
respondent, or to persons other than the sender, 
a reply is justified. One should be generous in 
applying this criterion, particularly if the ques- 
tionnaire fully meets the other nine criteria. 

4. Is the questionnaire well organized?— 
For a detailed discussion of this point see 
pages -15-6 and 20-2. If a questionnaire is 
well organized the fact is usually evident at 
a glance. The major and sub-items are logi- 
cally related and other indications of careful 
planning are evident. 

5. Are the questions clearly and briefly 
worded?—The length of questions will neces- 
sarily differ with the nature of the subject 
being studied. Pages 15-6 and 19-22 of this 
Bulletin will be suggestive in applying this 
criterion. 

6. Can most of the questions be briefly 
answered?—lInsofar as possible, questions 
should be answerable with a check mark or 
by a fact or a figure, and the number of ques- 
tions requiring extensive, subjective replies 
should be kept to a minimum. The reasons for 
this criterion are given on pages 18-21 and 36 
of this Bulletin. 

7. Is the information requested unavail- 
able?—Is it obtainable only through ques- 
tionnaire? The justification of this criterion is 
self-evident. It is discussed further on page 16. 

8. Is it set up in proper mechanical form?— 
The items which together constitute good me- 
chanical arrangement are discussed on page 22. 
Among the more important of these are neat- 
ness, easy legibility, sufficient space for answers, 
and proper identification. 

9. Are the demands reasonable?—This 


question needs to be answered both with +}), 
importance of the investigation and the 
spondent’s facilities for answering in mind 
More specific discussion of this point will] | 
found on pages 18-20. 

10. Is a proper return promised respond. 
ents ?—Questionnaires which benefit no one 
but the sender can seldom be justified. ‘This 
point is further elucidated on page 17. 

It is not suggested that a long and tiresome 
application of these criteria be made. It 
enough to have them in mind, or to have 
page 39 of this bulletin open before one 
he makes the decision “thumbs up” or “thumbs 
down”’ on a questionnaire. 

If yes can be answered to all or to most of 
the foregoing questions as a result of a brief 
inspection of a questionnaire, then it deserves 
further attention. If any considerable numbe: 
of the criteria have to be answered with a 
no, then the questionnaire should be 
promptly returned to the sender unanswered. 

The application of this rating plan need not 
be a one man performance. The one responsible 
for handling questionnaires in doubtful cases, 
should consult well-informed associates. lor 
example, the advice of the school accountant 
should be enlisted in deciding what question 
naires on school costs should be answered. 

In the case where it is decided that a ques 
tionnaire is not to be answered, it should be 
returned to the sender, along with a brief \et 
ter. It would be well to give the reason why 
the questionnaire was not answered. If this 
were generally done, rather than merely throw- 
ing unanswered blanks in the waste basket, 
the effect would be beneficial all around. 

When the decision is to answer, necessary 
data should be assembled from the best source 
the organization affords. One person should 
not attempt to supply answers to all questions. 
Rather he should be responsible for securing 
the data from the proper source or sources. 
Most of the information asked by proper!) 
prepared inquiries can be supplied by clerks. 
When one person in an organization is re- 
sponsible, the load can be properly dis- 
tributed, and a maximum result secured for a 
minimum of time and energy consumed. 

A copy of the data supplied in response to 
questionnaires should be kept on file. ‘This is 


[40] 


























Ol 


m 








SAD analeabl 9 














desirable in case a question later arises as to 
‘ust what information was furnished. Also the 
data assembled in replying to a questionnaire 
may later be needed for local use. 

[he foregoing paragraphs apply equally to 
all agencies which receive any considerable 
number of questionnaires, whether it be the 
offices of a superintendent of schools and his 
administrative and supervisory staff, a school 
of education, or other agency. Just how the 
foregoing proposals can be applied in any par- 
ticular situation will depend on circumstances. 

The policy and office plan for dealing with 
questionnaires must be determined by each 
agency for itself. Merely to storm at ques- 
tionnaires is not to solve the problem they in- 
volve. Thoughtful determination of policy, 
and efficient execution of the same is needed 
in dealing with questionnaires as in discharg- 
ing other duties. 

Need of a policy in sending out question- 
naires—The need for a policy and plan for 
dealing with incoming inquiries is matched by 
4 similar need in sending them out. The ma- 
terial in the sections of this bulletin beginning 
on pages 14 and 18 are particularly pertinent 
to all agencies which send out questionnaires. 
Too often questionnaires, both as to subjects 
covered and numbers dispatched, are out- 
growths of individual whims or evanscent 
needs rather than fulfillments of well-rounded 
policies in collecting information. This applies 
in some measure to school systems, schools of 
education, educational organizations, and other 
agencies which assemble educational data. 

In arriving at a policy and plan in sending 
out questionnaires, agencies such as these enu- 
merated might well take these steps: 

1. Once a year the whole matter should be 
a subject of discussion at a staff conference. 
The result will frequently be a plan which 
will involve the sending out of fewer and bet- 
ter questionnaires. As a basis for discussion 
there should be presented at this conference, a 
survey of the number, character, and outcome 
of those sent out by the organization during 
the previous year. 

2. Some simple machinery should be set up 
to guarantee: 


a. That all questionnaires are well pre 
pared and meet criteria such as those 
in the chapter on Better Preparation 
of Questionnaires. 

b. That too many questionnaires are not 
sent out, and particularly that unneces 
sary questions are avoided. 


That a permanent, continuing file of 


a 


all questionnaires sent out is kept in 
one place. 


Just how the foregoing will be worked out 
by any particular agency will depend upon ci: 
cumstances. In some cases the whole arrange 
ment for regulating the outgo from a school 
system, school of education, or educational as 
sociation should be on the informal, voluntary 
basis. In other cases it may need to be a rather 
definite scheme closely administered by the 
chief executive, or a member of his staff to 
which this responsibility is delegated. 

The chapter on Better Preparation of Ques 
tionnaires offers a number of suggestions which 
may well be consulted. Material on page 14 
is particularly pertinent to school systems. 

Questionnaires by college students—It is the 
practice of some departments of education to 
encourage students to send out questionnaires 
in connection with graduate theses and even in 
the preparation of term papers. Figures pre- 
sented elsewhere show that such inquiries con- 
stitute but a minor fraction of those which 
reach superintendents’ offices. But this small 
number has probably aroused more antagonism 
than all other questionnaires taken together. 
Evidence in the files of the Research Division 
shows that many questionnaires sent out by 
students are poorly conceived and miserably 
prepared. Furthermore, experience shows that 
even when the results obtained are actually 
summarized, it is often difficult to get a copy 
of this summary. The facilities which these 
students have are frequently inadequate eithe: 
to the proper preparation of forms or the han- 
dling of replies. 


This situation suggests that it would be a 
wise step if responsible colleges and universities 
would make it the rule that all questionnaires 
be sent out by members of the faculty, even 
though ‘some of the data assembled is used in, 
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theses.’ This practice would have two advan- 
tages. It would guarantee adequate supervi- 
sion. A faculty member would hesitate to have 
his name and official position attached to poorly 
conceived and poorly executed questionnaires. 
Second, it would eliminate the peculiar irrita- 
tion that many school people feel in being 
“asked by some unknown upstart to write 
his dissertation.” 

Probably the principal reason why ques- 
tionnaires from graduate students in some uni- 
versities have not been up to standard is the 
unreasonable number of studies assigned to 
faculty members for supervision. In some in- 
stitutions, the first step in guaranteeing ade- 
quate guidance of graduate students should be 
a reduction in the number of investigations as- 
signed to the individual professor for super- 
vision. 

Some institutions are finding it wise to pro- 
vide graduate students with specialized serv- 
ices of both a professional and clerical nature. 
A person expert in the editing of manuscripts 
could greatly improve the clarity and vigor of 
the prose in many dissertations. Students work 
more effectively if they may obtain expert help 
in making tabulations and calculations at a 
reasonable cost. When such facilities are pro- 
vided, the student not only works more effec- 
tively but there is also a partial lifting of the 
burden of supervision expected of the responsi- 
ble faculty member. This leaves him free to 
concentrate his guidance upon the broader 
aspects of the student’s investigation. 

In making the foregoing suggestions, it is 
not implied that the circulation of question- 
naires in school systems by college people is a 
one-sided question. It was fully recognized in 
an earlier section that an important part of 
the knowledge now available for the guidance 
of school people has come from questionnaires 
in the hands of competent investigators, who 
have usually been members of college faculties. 
It is also recognized that practical school work- 
ers, superintendents, supervisors, and teachers 
constantly call upon the colleges for service of 
various kinds. It is little wonder that the col- 
lege professor who donates a considerable por- 
tion of his time and energy in furnishing in- 





1 In reviewing the manuscript of this Bulletin, Professor Leonard V. asks this question: ‘Would it not be better to 
cloguemed faculty authorization a? Questionnaires emanating from Ce graduate departments of education rather than 
urging that the questionnaires be out by the faculty i el | =e altermative would be acceptable providing an individus 


member of the faculty as well a8 the aculty as a whole au’ 
tent person and organization should feel responsible. 





formation and guidance to school systems. 
sents their failure to reply to his well-prepa: 
questionnaire. 

The recognition of these facts, howe, 
does not call for a modification of the con: 
tion that all agencies which send out questi 
naires, colleges as well as others, may wit! 
benefit review their whole policy in this regard 
so that it may be well conceived, and so ¢| 
practices which discredit this method of 
quiry may be avoided. 

This section has outlined certain lines . 
voluntary, cooperative action which if followed 
by the various agencies which send out and re 
ceive educational questionnaires, will eliminate 
many of the abuses which have brought this 
legitimate method of investigation into disre 
pute. The next paragraphs offer an additiona! 
facility aimed at the same result. 

A clearing house for questionnaires—In thx 
two years that the Research Division has been 
studying the problem it has virtually become a 
clearing house for educational questionnaires. 

It is proposed that the Division undertake to 
continue this function on a permanent basis. 
In order to accomplish this result it would be 
necessary to have cooperation along these lines: 

1. Those who send out educational question- 
naires should send three complete copies of 
their form and letter of transmittal to the 
Research Division of the National Education 
Association, 1201 16th St., N. W., Washington, 
D. C. 

2. When the results of the study or reports based 
on the questionnaire are completed copies 
should be sent to the Research Division. 

3. When it is not practical to furnish a copy of 
the report, then notice should be sent to the 


Research Division as to where the results of 
the questionnaire are available. 


The questionnaires and reports so received 
will be properly classified, cross indexed, and 
filed. 

It would then be possible for the Division 
to organize a “Questionnaire Service” along 
these lines: 


1. Investigators contemplating the circulation of 
a questionnaire could be loaned copies of in 
quiries previously sent out on that topic. 

2. Investigators could be furnished a bibliog 
raphy of completed studies on the topic. 


questionnaire. The ee | principle is that some compe 


Any means which gets this result is acceptable. 
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The results of questionnaire investigation 
could be put into wider use, by annually issu- 
ing a summary of the results of “Question- 
naire Studies” for general circulation, and by 
citing these studies in answering inquiries for 
information. 


In developing the “Questionnaire Service’’ 
the Research Division: 


. Will not be able to assist investigators in 
drafting questionnaires, to criticize those 
drafted, nor to aid in the revision of question- 
naires proposed for circulation. These are func- 
tions which can be best exercised by the agency 
sponsoring the questionnaire. 

. Will not be able to sponsor nor circulate ques- 
tionnaires for investigators. The Division al- 
ready is called upon by the various Commit- 
tees and Departments of the National Educa- 
tion Association to circulate all the question 
naires it can wisely handle. Requests that the 
Division sponsor or circulate questionnaires 
which do not have to do with official projects 
of the National Education Association can- 
not be granted. 


By entering into an informal, voluntary plan 
for regulating educational questionnaires along 


the lines suggested above, school systems, 
schools of education, and education associations 
could remove most of the evils and preserve the 
good results which come from circulation of 
these inquiries. 

Summary—This chapter proposes an info1 
mal, voluntary plan for the regulation of ques- 
tionnaires in education. It is suggested that all 
agencies which receive questionnaires develop 
a policy and office plan for quickly identifying 
those worthy of attention and for economically 
providing the information they request. Prac- 
tical means of accomplishing these ends are 
outlined. 

It is also proposed that agencies sending out 
questionnaires develop unified policies which 
will result in the reduction of the number of in- 
quiries sent out and which will guarantee that 
those dispatched are well prepared. Certain 
definite suggestions for the attainment of these 
results are offered. 

A proposal is also made to organize a “Ques- 
tionnaire Service,” for the benefit of those who 
conduct or use the results of questionnaire in- 


vestigations. 








Some Sources of Information 


In a previous section of this Bulletin, it was 
urged that the decision to issue a questionnaire 
be reached only after a thorough canvass has 
been made of information already available. 
Duplication and waste of effort may thereby 
be avoided. This section calls attention to cer- 
tain, although not all, of the sources of in- 
formation which may be utilized by the in- 
vestigator in making such a canvass. 

1. The Office of Education of the United 
States Department of the Interior ‘\—(For- 
merly the Bureau of Education). This govern- 
mental agency is one of the principal store- 
houses of educational information. A letter ad- 
dressed to The Commissioner of Education, 
Office of Education, United States Depart- 
ment of the Interior, Washington, D. C., will 
usually bring pertinent material in bibliographic 
or other form on any educational question. The 
annual bibliography of research studies in edu- 
cation ? first issued in 1926-27 promises in time 
to furnish a complete and prompt round-up of 
educational investigations both in progress and 
completed. The regular record of current edu- 
cational publications * gives classified and an- 
notated references to certain educational period- 
icals and other publications. The annual Edu- 
cational Directory should be familiar to all 
who have to do with questionnaires.* These are 
but a few of many resources ° which the Office 
of Education has for aiding the investigator. 

2. The National Education Association— 
This professional organization has rapidly de- 
veloped its facilities in recent years and through 
its headquarters library, its Research Division 
which issues the Research Bulletin, its Division 
of Publications which issues the Journal and 
the annual volume of Addresses and Proceed- 
ings, and its various other divisions, is able 
either to furnish or to suggest sources of in- 
formation which may exist on most educational 
topics. The periodic round-up of city school re- 





search studies, completed and in progress,° the 
annual summary of educational legislation 

its questionnaire service and other similar + 

sources might appropriately be consulted ear|, 
by most investigators. It regularly compile: 
unique information in several fields amony 
which are curriculum reconstruction, the work 
of the principal, state educational legislation. 
teacher salaries and retirement systems. Ad- 
dress the Association at 1201 16th St., Was) 
ington, D, C. 

3. The Education Index—This publication 
may be consulted in well-equipped educationa| 
libraries. It was begun in 1929, replacing th: 
Loyola Educational Index, which covered 
1928. It indexes more than a hundred educa 
tional periodicals and list books and materia! 
in pamphlet form. The expansion of the cove: 
age of this publication and the cumulative valuc 
of its issues will rapidly provide an indispensa 
ble tool for educational research workers. (H. 
W. Wilson Co., 958 University Ave., New 
York City.) 

4. Readers’ Guide to Periodical Literatur: 
—Indexes, among other magazines, ten wel! 
known educational magazines, the bulletins of 
the United States Office of Education, and the 
Proceedings of the National Education Asso 
ciation. Available at libraries. Published 
monthly ; began in 1900. (H. W. Wilson Co.., 
958 University Ave., New York City.) 

5. International Index to Periodicals 
(Formerly the Readers’ Guide Supplement ) 
Indexed through 1928 nearly a score of thx 
technical and specialized educational maga- 
zines, and a small number of the foreign 
periodicals emphasizing education. Published 
five times yearly; began in 1907. The educa- 
tional magazines were transferred to the Edu- 
cation Index on January 1, 1929. (H. W. 
Wilson Co., 958 University Ave., New York 
City.) 


i* ‘United States Office of Education.” Journal o* the National Education Association 18: 241; November, 1929. 
2 United States Ay +t! the Interior, Bureau of Education. ae of Research Studies in Education, 1926-2 
Bulletin No. 22, 1928. Printing Office, Washington, D. C. 162 
2 United States woh a t of the Interior, Bureau =. — Record of Current Educational Publications. Bulletin No 
23, 1928. ted States De 


Hr aT Washington, D. C. 1 
5 For example, see 


* National Education Association, ——¥* ¥. — 
April, 1929. Circular No. $, 1929. Washington, D. C 


of the Interior, Bureau of Bulletin (1929) No. 1. Educational Directory: 1929. 
CE. and Greenleaf, W. J. Bibliography of 
Department of the Interior, Bureau of Educati tion, Bulletin No. 2 
aes. City School Research y4s Completed and in Progre 


Secondary Beaten paren. 1920-1925. United Stat: 
1926. Washington, D. C 


7 National Education Association, Research Division’ State School Legislation, 1929. Washington, D. C. Mimeographed, 5 


p., 25c. 
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6. The United States Catalog: Books in 
Print January 1, 1928—Supplemented by 
monthly issues of the Cumulative Book Index. 
An index, by author, title, and subject, of ail 
hooks in print in the United States; includes 
many pamphlets and local governmental re- 
ports. Available at libraries. (H. W. Wilson 
Co., 958 University Ave., New York City.) 

7. Psychological Index—Annual bibliog- 
raphy of the literature of psychology and cog- 
nate subjects in the United States and abroad. 
Most recent volume issued June, 1929, con- 
tains 408 pages. (Psychological Review Co., 
Princeton, N. J.) 

8. The Loyola Educational Digest—Con- 
sists of digests of articles and bulletins, issued 
in card form, adapted for library filing. Pub- 
lished monthly, except in July and August; 
began in 1924. (Loyola Educational Digest, 
3441 North Ashland Ave., Chicago, III.) 

9. Titles of Masters’ and Doctors’ Theses 
in Education Accepted by Colleges and Uni- 
versities in the United States—A series of re- 
ports begun in 1917. Issued biennially; latest 
number covers the period ending October 15, 
1927. (College of Education, University of 
Illinois, Urbana, III.) 

10. Ten Years of Educational Research, 
1918-1927—A bibliography of nearly 3,700 
reports of educational research, including 
periodical references, reports, and doctors’ 
theses. Compiled by Walter S. Monroe and 
others, published in 1928 by the University of 
Illinois, Urbana, Ill. This is illustrative of the 
type of valuable reference material of occa- 
sional issuance by various educational agencies 
which a search will reveal. 

11. Educational Research: Suggestions and 
Sources of Data with Specific Reference to 
Administration—A 77-page pamphlet by 
Carter Alexander, published in 1929 by the 
Bureau of Publications of Teachers College, 
Columbia University, New York City. Calls 
attention to bibliographies and other sources of 
information on some 300 individual items, 
alphabetically arranged. 

12. Teachers Journal and Abstract—-Each 
number contains from sixty to eighty signed 
abstracts of articles in educational magazines 
of the previous month. A consistent plan of 
classification is followed. Published monthly, 


except July and August, by Colorado State 
Teachers College, Greeley, Colo. 

13. Continuing Studies—In a few major 
fields of the school curriculum a definite plan 
ot summarizing research has been developed. 
Investigators in the subject fields mentioned 
below will do well to consult the following 
sources of information before undertaking new 
studies. 

a. Arithmetic. Investigations in the field of 
arithmetic are summarized annually by Guy T. 
Buswell and published in the Elementary 
School Journal. These annual studies supple- 
ment a monograph published in 1925, and 
taken together present a complete survey of 
research in arithmetic. The first and the most 
recent compilations follow: 


BUSWELL, Guy THOMAs, and Jupp, CHARLEs Hus- 
BARD. Summary of Educational Investigations 
Relating to Arithmetic. Supplementary Educa- 
tional Monographs, No. 30. University of Chi- 
cago Press, Chicago, Ill. 1926. 212 p. 

BuUSWELL, Guy THOMAS. “Summary of Arithmetic 
Investigations (1928) I-II.” Elementary School 
Journal 29: 691-8, 737-47; May-June, 1929 


b. Grammar, Language, and Composition. 
A bibliography of 264 selected titles has been 
classified and reviewed by R. L. Lyman in the 
following monograph, which will doubtless 
be followed by annual supplements : 


LyMAN, R. L. Summary of Investigations Relating 
to Grammar, Language, and Composition. Sup- 
plementary Educational Monographs, No. 36. 


, 


University of Chicago Press, 1929. 302 p 


c. Reading. An annual summary of reading 
investigations is published by William Scott 
Gray in the Elementary School Journal. ‘These 
reports are supplementary to an exhaustive 
monograph issued in 1925, and titled as fol- 
lows: 


Gray, WILLIAM Scott. Summary of Investigations 
Relating to Reading. Supplementary Educational 
Monographs, No. 28. University of 
Press, Chicago, Ill. 1925. 275 p. 


Chicago 


The most recent survey, containing 101 


titles, is listed below: 


Gray, WiLtiAM Scorr. “Summary of Reading In- 
vestigations (July 1, 1927 to June 30, 1928) 
I-II.” Elementary School Journal 29: 443-57, 
496-509; April, March, 1929. 


d. Science. Digests of 70 learning and cur- 
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ricular studies in science published prior to 
1925 have been brought together by Curtis in 
the following book, which is to be brought up 
to date in future volumes: 


Curtis, Francis Day. Digest of Investigations in 
the Teaching of Science in the Elementary and 
Secondary Schools. P. Blakiston’s Sons and Co., 
Philadelphia, Pa., 1926. 341 p. 


e. Secondary-school administration. ‘The 
Department of Secondary-School Principals of 
the National Education Association publishes 
from time to time digests of outstanding re- 
search studies in that field. Their latest com- 
pilation is the following: 


Reavis, WittiamM C. and Burtscu, Russet L. §., 
“Abstracts of Unpublished Masters’ Theses in 
the Field of Secondary-School Administration.” 
Bulletin of the Department of Secondary School 
Principals of the National Education Association, 
Bulletin, No. 24, January, 1929, Cicero, Ill. 202 p. 


f. Social sciences. Abstracts of studies in the 
social sciences were presented by Earle U. 
Rugg in the Third, Fifth, and Sixth Year- 
books of the Department of Superintendence 
of the National Education. The latest com- 
pilation is presented in the following volume: 


Ruocc, Earte Unperwoop, Curriculum Studies in 
the Social Sciences and Citizenship, Colorado 
Teachers College Education Series, No. 3, Colo- 
rado State Teachers College, Greeley, Colorado, 
1928. 214 p. 


14. American Council on Education—This 
agency is able to provide information relating 
to higher education. Below are examples of the 
type of information which the Council makes 
available. Its headquarters are at 26 Jackson 
Place, Washington, D. C. 


Rosertson, Davin A. American Universities and 
Colleges. Published for the American Council on 
Education, by Charles Scribner’s Sons, New 
York. 1928. 884 p. 


15. Professional Organizations—Many edu- 
cational associations in the United States spon- 
sor research. A list of all educational associa- 
tions is published each year in the Educational 
Directory of the U. S. Office of Education. 
The most recent issue of the directory ap- 
peared in May, 1929. It may be obtained only 
through purchase (30 cents a copy) from the 


Superintendent of Documents, Governm 
Printing Office, Washington, D. C. 

The following associations are typical 
sources of data on subjects frequently cove: 
by questionnaires: 

International Kindergarten Union, |. 
16th Street, N. W., Washington, D. C. 

National Society for the Prevention 4; 
Blindness, 370 Seventh Avenue, New York 
City. 

American Child Health Association, 
Fifth Avenue, New York City. 

16. National Surveys—Studies which «: 
tempt broadly to survey the situation in certain 
major educational fields are growing in num 
ber and offer an essential background for {u 
ture investigations. Among these are: 

a. Modern Foreign Language Study. This 
thorough-going investigation, in cooperation 
with the Canadian Committee on Foreign 
Languages, has published a number of re. 
searches and has others in preparation. As one 
of the recent publications, the following bul- 
letin is mentioned : 


Purin, C. M. The Trainina of Teachers of th 
Modern Foreign Languages. Publications of the 
American and Canadian Committees on Modern 
Languages, Vol. 13. Macmillan Co., New York 
1929. 111 p. 


Inquiries should be addressed to the Modern 
Foreign Language Study, 561 West 116th 
Street, New York City. 

b. The United States Office of Education 
now has under way studies of the Land Grant 
Colleges, of the relation of the federal govern- 
ment to education and of secondary education. 
It has also been proposed that this office or- 
ganize a survey of teacher training and schoo! 
finance. 


17. Yearbooks of Various Organizations 
A rapidly growing number of educational or- 
ganizations issue annual volumes which con- 
tain valuable material. Among these are: 


a. The Departments of Superintendence, of 
Elementary School Principals, of Classroom 
Teachers, etc., of the National Education As- 
sociation, 1201 Sixteenth St., Washington, 
D.C. 

b. The National Society For the Study of 
Education. Public School Publishing Com- 
pany, Bloomington, Illinois. 
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[he National Society of College Teach- 
.§ Education. University of Chicago Press, 
cago, IIL. 
d. The Educational Press Association. The 
Fifth Yearbook 1929, lists and classifies sev- 
hundred periodicals dealing primarily 
with education. 1201 Sixteenth St., Washing- 
ton, D. c. 
The foregoing sources of information .are 
illustrative rather than complete. They are 


enough to support the statement that it 1s 
wise to discover what is already available be 
fore one seeks information in any field. Even 
though agencies such as the United States 
Office of Education or the National Educa 
tion Association may not be able directly to 
supply information on an educational topic, 
they may be able to put the investigator in 
touch with an agency which can provide part 
or all of what is desired. 


References on the Use of Questionnaires in Educational Research 


From a much larger number of references 
which directly or indirectly bear upon the 
proper use of questionnaires in education, the 
following have been selected and briefly an- 
notated. Many of those cited are worthy of a 
complete reading, particularly by those who 
deal with questionnaires either as investigators 
or respondents. 


1. ALEXANDER, CARTER. School Statistics and Pub- 
licity. Silver, Burdett and Company, Boston, 
1919. p. 59. 

In general discourages use of questionnaires 
for gathering statistics, because of factors of 
selection, and unreliability of answers. 

. Bawven, W. T. “Searching After the Truth.” 
Industrial Education Magazine, 27: 172-3; De- 
cember, 1925. 

Commends the questionnaire as an important 
and valuable method of investigation, 
used by responsible agencies. Urges that per- 
sons inform themselves as far as possible re- 
garding the subject of their inquiry before issu- 
ing a questionnaire. 

. Brxter, Harotp Hencn. Check Lists for Edu- 
cational Research. Bureau of Publications, 
Teachers College, Columbia University, New 
York City. 1928. p. 40-5. 

Lists forty-six criteria which should govern 
the preparation and sending of questionnaires. 

. Borinc, E. G. “When and How to Reply to a 
Questionary.” American Journal of Psychology 
37: 632-3, October, 1926. 

Quotes a “code of action” for use in replying 
to questionnaires, drawn up by a group of 
scientists in four of the larger psychological 
laboratories. 

. BuckincHaM, B. R. “The Questionnaire.” Jour- 
nal of Educational Research 14: 54-8; June, 
1926. 

Urges that the investigator assure himself 
(1) that he cannot collect his data except by 
questionnaire, and (2) that the problem is not 
trivial. States that the recipients owe their 
assistance to the cause of education. 


when 


6. Burk, Freperic. “On a Certain Questionnaire 

School and Society 15: 170-3; February 11, 1922. 

A protest against a particular questionnaire 
seeking data on degrees, teacher load, etc., and 
criticises the educational ideals which prompted 
the questionnaire. 
BUTTERFIELD, E. W. 
of the Dotted Line.’ 
1-4; January, 1925, 

Criticises the questionnaire method in con 
ducting the Classical Investigation. 

. BUTTERFIELD, E. W. 
the Questionnaire.” School and Society 11 
January 10, 1920. 

Outlines a suggested policy for replying to 
questionnaires, 

. CHarters, W. W. Curriculum Construction 
Macmillan Co., New York. 1924. p. 133-6. 

Points out five major limitations of the ques 
tionnaire. Recommends preliminary trials of the 
form, the use of check questions, and the sub 
stitution of the oral interview whenever 
sible. 

. Crawrorp, CLAupE C. The Technique of Re 
search in Education. University of Southern 
California, Los Angeles, California. 1928. Chap 
ter X, “Questionnaire Technique,” p. 177-87 

Gives specific suggestions; emphasizes pre 
liminary considerations; points out 
faults, such as ambiguity, and leading questions. 
Douctas, H. R. “The Questionnaire—To Be o1 
Not To Be.” School and Society 15: 397-9; 
April 8, 1922. 

A defense of the use of questionnaires, when 
properly prepared, and used sparingly. 

12. FaALton, Joun F. “The Questionnaire in Edu- 
cational Research.” Catholic Educational Re 
view 23: 539-45; November, 1926. 

Reviews the history of questionnaires ; digests 
the literature concerning them. Defines ques 
tionnaire as: “a printed form presenting for 
answer a number of questions on a single topic, 
or a group of closely connected topics arranged 
under separate heads.” States that when com- 
bined, with other methods and used by skilled 
workers the questionnaire is of real service. 


“The Plenary Inspiration 


Educational Review 69: 


J 


Ethics and 


55-6; 


“Professional 


pos- 


common 
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17. 


19. 


20. 


. Gauct, R. H. “History of Questionnaire in 


Method of Research.” Pedagogical Seminary 
14: 366-83. September, 1907. 

Mentions first written questionnaire known, 
issued by Statistical Society, of London, in 
1838, on the history of strikes. Refers to work 
of G. Stanley Hall, 1880, in using questionnaires 
to study the mental content of children entering 
school. 


. Goon, Carrer. How to Do Research in Educa- 


tion. Warwick and York, Inc., Baltimore. 1928. 
p. 133-40. 

Summarizes published discussion of the ques- 
tionnaire method in educational research; 
quotes Buckingham, Wylie, and National Com- 
mittee on Research in Secondary Education. 


. HANKINSON, FRANK. “The Blight of the Ques- 


tionnaire.” Educational Review. 73: 
February, 1927. 

Discusses the ambiguity of questionnaires, 
endorses the policy of not answering them at all. 


102-8; 


. Jones, ARTHUR J., Chairman National Com- 


mittee on Research in Secondary Education. 
An Outline of Methods of Research with Sug- 
gestions for High School Principals and Teach- 
ers. U. S. Department of the Interior, Bureau 
of Education. Bulletin, 1926, No. 24, Govern- 
ment Printing Office, Washington, D. C. 31 p. 

Section V, “The Questionnaire,” gives specific 
directions and suggestions for the use of the 
questionnaire method. 
Kevttey, TruMan L. Scientific Method—lIts 
Function in Research in Education. The Ohio 
State University Press, Columbus, Ohio, 1929. 
195 p. 

Lecture II, pages 33 to 83, is particularly 
pertinent. 


". Koos, L. V. The Questionnaire in Education. 


Macmillan Co., New York. 1928. 178 p. 

A comprehensive critique and manual on 
questionnaires. Reports analysis of a large 
sampling of published questionnaire investiga- 
tions; evaluates the questionnaire as a method 
of investigation; and makes recommendations 
as to desirable practices in issuing question- 
naires. Selected bibliography deals with ques- 
tionnaire method, and cites representative stud- 
ies which have employed this method of investi- 
gation. 

Linpsay, E. E. “Questionnaires and Follow-up 
Letters.” Pedagogical Seminary 28: 303-7; Sep- 
tember, 1921. 

Reports experiments in which follow-up letters 

brought in a higher number of replies than the 
original inquiry. 
Mann, Horace. Life and Works of Horace 
Mann. Lee and Shepard, Boston, Massachusetts, 
1891. Vol. III, p. 97-109, 157-67; Vol. IV, p. 
151-71. 

Quotes circular letters and replies received, 
issued in 1841, 1842, and 1847. One was sent to 
employers, on the influence of a public school 
education on the earnings of laborers; one was 


{ 


24. 


27. 


28. 


29. 
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sent to teachers on the moral effects of » 
schools, and one to doctors on the proportioy 
disease due to ignorance of the laws of }h, 


. Matuews, C. O. “The Effect of the Ord, ¥ ‘ 


Printed Response Words on an Interest Que. 
tionnaire.” Journal of Educational Psycho), 
128: 34, February, 1929. 

Reports a tendency to give higher percent 0; 
affirmative replies to item printed on the ex 
treme left position on the page. 


. “QUESTIONNAIRES AND QUESTIONNAIRES.” J m>) 


ican School Board Journal 71: 74; August, 1925 

Says that the average school board in a cir, 
of 25,000 population receives at least one « 
tionnaire each day of the school year; empha 
sizes the part played by the graduate student 
distributing questionnaires. 


. Reever, Warp G. How to Write a Thesis. Pub 


lic School Publishing Company, Bloomington 
Illinois. 1925. p. 34-8. 

Mentions ten essential considerations in «: 
signing and distributing questionnaires. 

Rucc, Haroip O. Statistical Methods Applied 
to Education. Houghton Mifflin Company, Bos 
ton. 1917. p. 40-55. 

Contains section on the question-blank method 
of collecting educational data. Classifies ques 
tionnaires by three types, those seeking data 
(1) that can be answered from persona! in 
formation, (2) that can be found in schoo! re 
ords, (3) that represent analyses and judg- 
ments. Gives essential steps and guiding prin 
ciples in use of questionnaire method. 


. ScHiuter, W. C. How to Do Research Hori 


Prentice-Hall, Inc., New York. 1926. p. 84-7 

States that a proper use of questionnaires can 
be measured by the number of completed 
turns. Makes suggestions as to preparation 
questionnaires. 


. Symonps, Perciva, M. Measurement in Second 


ary Education. Macmillan Co., New York. 192 
Chapter 16, “The Measurement of Conduct 
p. 333. 

Refers to work by Voelker, Cudy, Woodworth 
and Raubenheimer in using questionnaires « 
study character traits. 

Symonps, PercivaL M. “Method of Investiga 
tion of Study Habits.” School and Society 2+ 
145-52; July 31, 1926. 

Lists twelve investigations of study habits by 
use of questionnaires; points out weaknesses o! 
the method for this purpose. 

Toors, Herspert A. “Validating the Question- 
naire Method.” Journal of Personnel Researc/ 
2: 153-60; August-September, 1923. 

Describes a study in which a 93 percent re- 

turn to a questionnaire was obtained through 
the use of 5 follow-up letters. 
Toops, Herpert A. “Returns From Follow-up 
Letters to Questionnaires.” Journal of Applied 
Psychology 10: 92-101; March, 1926. 

Reports a 100 percent return on a question- 











, 


naire, through the use of 6 follow-up letters. 
Cumulative percent of response as follows: 
original questionnaire, 53%, follow-up letters, 
65%, 83%, 88%, 97%, 99%, 100%. Observes 
tendency of recipients known to senders to reply 
more promptly than strangers. The essential 
conclusion of the study materially 
changed through the replies obtained by follow- 


was not 


up letters. 


0. Trow, W. C. Scientific Method in Education. 


Houghton Mifflin Company, Boston, 1925. 
Points out weaknesses of questionnaire 
method in child psychology. 


31. WuippeLe, Guy M. “The Improvement of Edu- 


cational Research.” School and Society 26: 249- 
59; August 27, 1927. 

The writer sets forth some of the ways in 
which educational research falls short of be- 
ing truly scientific in spirit and practice. States 
characteristics of a good questionnaire. 


. Wurrney, F. 


33. WyYLie, 


P. “Questionnaire Craze.” Educa 
tional Review 68: 139-40; October, 1924. 

“The results of questionnaires as usually in- 
terpreted tend but to strengthen the hold of 
mediocrity.” Admits that we sometimes 
need to know what the other fellow is doing, 
but would discourage the attempt to establish 
norms and averages, through the use of ques 
tionnaire methods. 
AnpREW T. “To What May 
We Rely Upon the Answers to a School Ques- 
tionnaire?” Journal of Educational Method 6: 
252-7; February, 1927. 

Refers to questionnaires to school children. 


Extent 


Says may be relied provided 
(1) that questions are plainly 
that there are a large number of interlocking 
questions, and (3) that the number of individ- 


uals questioned is large enough to balance the 


answers upon, 


worded; (2) 


errors. 


FORM I.—POSTAL CARD USED BY COOPERATING SCHOOL SYSTEMS IN RE- 


PORTING RECEIPT OF QUESTIONNAIRES. 








A questionnaire has been received at this office from: 





SCHOOL OF STREET 


Y ano stare 


Subject of questionnaire: 


Date of questionnaire: 


This information is furnished to the Research Division of the National 
Education Association through the office of the Superintendent of 


Schools in:_ 


\ERACT NAME OF TITLE 














FORM II.—FORM 





USED IN COLLECTING INFORMATION ON PERCENT OF 
RETURN ON QUESTIONNAIRE 











1. Have the replies to the above questionnaire been tabulate Yes N 


THE NATIONAL EDUCATION ASSOCIATION 
OF THE UNITED STATES 
$201 SIXTEENTH STREET 


WASHINGTON. OC 


March 29, 1929. 


Study of Questionnaires 


You will recall that last year you sent us @ copy of your questionnaire on 


Will you now give us the information requested below, concerning the results of 
your guestionneire? 








2. What uses have been made of the study resulting from the above questionnaire? 


(Please check below) 


a. Report on a local school problem........... ( 
bd. Report on problem of state school system... ( 
0. Report for state or national organization . ( 
4. Graduate thesis .... 2.2... ccc cece cece eeeees 


@. College paper or report, not a thesis ..... ( 
f. Speeoh or leoture .... 2.26... ccc cc eneeee ( 
@- Magazine article ....... 0.6... ccc cece eeeee ( 
ie ME AMG RAE Rsopedeecercccgeccereceesess ( 
Be EE aa kncencctescercssccesscesienes ( 


j. Other use (please state) 


: 
| 


SS 








We TR Gg GB BOD cc cccescccccccccccccccccccges ( 


— 
oe 
~— 


3. In what mechanical form is the study resulting from the above questionnaire, at 


i present? (Please check below) 
if @. Handwritten (____) 4. Printed 8 
bd. Typewritten (___) e. ( ) 


a oe ee 





©. Mimeographed (____) 


We should be glad to receive a copy of the study resulting from your question- 


i naire if available. 


naires. 








H Please return one copy of this letter, keeping the other for your files. Your 
P name is on our mailing list to receive a copy of the report on our study of question- 


Joth K. Horton 
Direotor, Research Division 
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